-- Logs begin at Fri 2022-09-09 11:25:30 UTC, end at Fri 2022-09-09 12:37:52 UTC. -- Sep 09 11:51:36.992749 dv-cinder-852210-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 09 11:51:37.000169 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 9 11:51:36 2022] *** Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: nodename: dv-cinder-852210-13-iscsi Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: machine: x86_64 Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: clock source: unix Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: pcre jit disabled Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: detected number of CPU cores: 4 Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: current working directory: / Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: detected binary path: /usr/bin/uwsgi-core Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: your processes number limit is 47680 Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: your memory page size is 4096 bytes Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: detected max file descriptor number: 2048 Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: lock engine: pthread robust mutexes Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: thunder lock: enabled Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 09 11:51:37.004841 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 09 11:51:37.044126 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: Python main interpreter initialized at 0x5589406039c0 Sep 09 11:51:37.044126 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: python threads support enabled Sep 09 11:51:37.044126 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: your server socket listen backlog is limited to 100 connections Sep 09 11:51:37.044126 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: your mercy for graceful operations on workers is 90 seconds Sep 09 11:51:37.044521 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: mapped 403077 bytes (393 KB) for 2 cores Sep 09 11:51:37.044601 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: *** Operational MODE: preforking *** Sep 09 11:51:37.044850 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: *** uWSGI is running in multiple interpreter mode *** Sep 09 11:51:37.044850 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: spawned uWSGI master process (pid: 81433) Sep 09 11:51:37.045059 dv-cinder-852210-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 09 11:51:37.045634 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: spawned uWSGI worker 1 (pid: 81447, cores: 1) Sep 09 11:51:37.045969 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: spawned uWSGI worker 2 (pid: 81448, cores: 1) Sep 09 11:51:37.048793 dv-cinder-852210-13-iscsi devstack@keystone.service[81433]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 09 11:51:41.370906 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 11:51:41.370906 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: from cryptography.utils import int_from_bytes Sep 09 11:51:41.371694 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 11:51:41.371694 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: from cryptography.utils import int_from_bytes Sep 09 11:51:41.372137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 11:51:41.372137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: from cryptography.utils import int_from_bytes Sep 09 11:51:41.372270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 11:51:41.372270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: from cryptography.utils import int_from_bytes Sep 09 11:51:41.421988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 11:51:41.422203 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 11:51:41.422299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 11:51:41.422393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 11:51:41.422476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] command line args: [] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 11:51:41.422556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] command line args: [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 11:51:41.422638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 11:51:41.422754 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 11:51:41.422873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ================================================================================ {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 11:51:41.422945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ================================================================================ {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 11:51:41.423081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] admin_token = **** {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.423187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] admin_token = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.423271 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] config_dir = [] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.423414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] config_dir = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.423511 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.423593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.423674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] config_source = [] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.423758 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] config_source = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.423837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] debug = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.423949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] debug = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.424049 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.424226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.424380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.424458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.424524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.424588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.424780 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.424916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.425054 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.425138 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.425260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] list_limit = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.425348 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] list_limit = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.425448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] log_config_append = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.425535 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] log_config_append = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.425604 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.425667 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.425785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] log_dir = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.425860 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] log_dir = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.425965 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] log_file = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.426049 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] log_file = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.426233 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.426316 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.426460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.426555 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.426711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.426808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.426958 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.427042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.427162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.427293 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.427389 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.427473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.427550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.427631 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.427785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.427899 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.428030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.428117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.428254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.428340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.428490 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.428570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.428723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.428856 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.428973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.429053 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.429219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.429297 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.429523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.429614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.430019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.430106 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.430216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] publish_errors = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.430290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] publish_errors = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.430453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.430585 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.430672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.430751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.430876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.430954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.431072 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.431165 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.431311 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.431393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.431547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] standard_threads = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.431625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] standard_threads = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.431778 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.431867 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.432021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.432100 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.432196 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.432275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.432397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] use_journal = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.432478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] use_journal = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.432552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] use_json = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.432637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] use_json = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.432711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] use_stderr = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.432792 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] use_stderr = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.432872 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] use_syslog = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.433000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] use_syslog = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.433078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.433153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:51:41.433329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.433409 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.433571 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.433652 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.433831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.433911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.434072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.434204 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.434349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.434436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.434593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.434677 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.434840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.434925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.435079 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.435162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.435329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.435409 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.435579 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.435657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.435830 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.435909 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.436068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.436147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.436315 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.436400 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.436553 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.436645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.436798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.436877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.437046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.437132 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.437283 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.437370 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.437539 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.437618 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.437790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.437869 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.438040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.438146 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.438301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.438391 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.438556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.438643 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.438817 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.438898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.439071 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.439150 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.439319 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.439407 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.439554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.439639 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.439784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.439869 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.440017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.440104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.440252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.440340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.440504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.440616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.440752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.440844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.440983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.441063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.441213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.441293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.441461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.441548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.441707 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.441792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.441946 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.442031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.442221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.442321 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.442458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.442537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.442695 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.442780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.442933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.443021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.443165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.443244 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.443399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.443483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.443630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.443780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.443855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.443927 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.444090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.444171 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.444340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.444428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.444577 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.444658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.444815 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.444895 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.445074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.445162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.445314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.445401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.445552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.445633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.445803 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.445920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.446027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.446185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.446292 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.446391 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.446517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.446646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.446749 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.446846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.447001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.447128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.447218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.447337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.447436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.447560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.447688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.447768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.447924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.448005 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.448163 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.448247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.448398 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.448530 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.448657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.448759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.448896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.448978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.449129 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.449209 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.449372 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.449471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.449613 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.449699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.449878 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.449969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.450163 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.450245 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.450377 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.450498 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.450612 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.450693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.450851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.450940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.451090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.451171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.451325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.451406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.451574 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.451657 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.451814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.451897 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.452053 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.452134 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.452300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.452379 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.452538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.452619 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.452784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.452872 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.453023 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.453112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.453259 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.453341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.453496 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.453583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.453734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.453815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.453973 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.454055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.454258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.454341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.454506 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.454589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.454759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.454842 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.455010 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.455098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.455254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.455335 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.455484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.455562 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.455726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.455808 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.455966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.456053 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.456205 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.456321 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.456434 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.456517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.456666 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.456746 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.456894 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.457030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.457193 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.457312 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.457392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.457466 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.457579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.457670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.457815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.457895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.458060 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.461838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.461838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.462174 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.462174 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.462502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.462597 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.462671 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.462754 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.462860 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.462944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] auth.external = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.463028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] auth.external = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.463104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.463165 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.463223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.463306 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.463383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.463464 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.463539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] auth.password = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.463619 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] auth.password = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.463694 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] auth.token = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.463766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] auth.token = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.463839 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.463920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.463995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.463995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.464126 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.464205 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.464205 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.464345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.464421 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.464494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.464567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.464638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.464710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] credential.caching = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.464793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.464867 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.464940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] credential.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465229 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465408 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.465959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.466019 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.466084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.466187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.466254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.466314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.466387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.466449 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.466508 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.466567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.466626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.466684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.466751 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.466817 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] federation.caching = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.466943 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.467016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.467187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.467258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.467431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.467503 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.467661 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.467730 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.467909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.467989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.468150 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.468220 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.468382 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.468452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.468634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.468711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.468874 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.468944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.469119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.469188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.469363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.469440 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.469600 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity.caching = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.469670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.469851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.469926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.470100 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.470300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.470367 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.470425 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.470590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.470661 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.470839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.470916 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.471070 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.471145 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.471310 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.471380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.471559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.471639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.471795 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.471864 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.472035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.472104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.472271 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.472349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.473195 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.473351 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.473482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.473611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.473611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.473864 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.473994 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.474169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.474308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.474437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.474565 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.474699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.474828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.474955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.475098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.475232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.475359 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.475494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.475494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.475732 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.475859 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.475986 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.476120 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.476256 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.477025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.477122 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.477315 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.477451 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.477589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.477725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.477851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.477977 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.478103 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.478251 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.478585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.478690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.478767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.478839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.478911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.478982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.479054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.479133 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.479207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.479279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.479350 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.479421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.479500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.479573 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.479645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.479717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.479788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.479860 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.479938 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.480012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.480090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.480163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.480236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.480307 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.480307 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.480444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.480516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.480587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.480659 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.480730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.480730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.480868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.480940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.481011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.481082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.481153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.481311 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.481394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.481551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.481632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.481792 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.481872 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.482033 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.482174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.482291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.482622 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.482788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.482868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.483046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.483126 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.483280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.483360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.483523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.483602 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.483771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.483851 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.484013 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.484091 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.484248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.484334 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.484480 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.484569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.484719 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.484800 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.484962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.485042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.485196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.485275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.485427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.485505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.485672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.485750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.485918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.486050 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.486173 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.486256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.486424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.486503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.486673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.486762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.486906 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.486986 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.487158 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.487237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.487399 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.487478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.487632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.487710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.487869 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.487953 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.488109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.488189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.488350 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.488496 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.488574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.488753 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.488851 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.488927 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.489057 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.489128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.489287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.489365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.489535 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.489616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.489941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.490077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.490247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.490371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.490371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.490621 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.490747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.490870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.490993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.491115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.491247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.491369 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.491492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.491627 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.491752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.491876 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] resource.caching = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.492519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.492602 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.492664 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.492722 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.492780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.492837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.492904 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.492970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.493153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.493283 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.493420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.493581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.493719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.493842 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.493967 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.494091 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.494246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.494415 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.494493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.494554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] role.caching = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.494612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.494670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] role.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.494828 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.494964 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.495106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] role.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.495232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.495359 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] role.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.495485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.495610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.495748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.495874 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.496003 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.496128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.496255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.496382 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.496521 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.496644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.496779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.496904 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.497033 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.497033 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.497273 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.497395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.497521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.497645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.497767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.497898 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.498031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.498237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.498366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.498366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.498609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.498733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.498870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.498995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.499118 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.499242 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.499365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.499684 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.499826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.499826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.500066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.500195 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.500324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.500468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.500595 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.500722 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.500848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.500975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.501101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.501237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.501364 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.501493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.501621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.501748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.501874 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.502009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.502158 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.502284 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.502413 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.502541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.502668 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.502806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.502891 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.502954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.503076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.503199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.503327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.503463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] token.caching = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.503591 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.503728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.503854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.503982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.504109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.504243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.504373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.504500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.504628 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.504744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.504868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.504999 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.505124 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.505247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.505247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.505489 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.505612 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.505612 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.505854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.505977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.505977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.506333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.506539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.506539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.506943 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.507144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.507364 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.508201 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.508341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.508481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.508618 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.508748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.connection = **** {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.508876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.509003 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.509132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.509259 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.509355 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.509523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.509653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.509653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.509946 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.510197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.510197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.510573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.510718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.connection = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.510847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.510975 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.510975 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.511216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.511343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.511480 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.511608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.511737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.511737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.511976 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.512103 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.512103 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.512364 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.512493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.512493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.512734 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.512862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.512997 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.513082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.513082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.513082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.513624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.513830 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 11:51:41.514039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.514039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.514039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.514039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.514848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.514848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.514848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.514848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.515559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:51:41.515559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 11:51:41.543651 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.543923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.543923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.544177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.544326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.544457 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.552580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.552768 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.582797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.583052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.583198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.583342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.638447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.638896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.687944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.688183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.688720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.688957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.845143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.845406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.845550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.845697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.845855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.845991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.846147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.846298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.846433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.846577 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.846712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.846712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.870536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.870770 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.870910 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.871065 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.871202 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.871354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.871483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.871617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.871758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.871890 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.872021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.872150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.872281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.872281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.883239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.883730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.896100 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.896325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.896548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.896710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.896851 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.896982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.897112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.897240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.897378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.897507 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.897636 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.897775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.897775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.898032 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81447) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:51:41.914627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.914814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.914961 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 11:51:41.915102 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 11:51:41.915239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.915390 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.915523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 11:51:41.915656 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 11:51:41.915807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.915941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.916082 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.916215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.916348 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.916481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.916567 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.916740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.925943 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.926082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.926246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.926392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.926527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.926671 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.926803 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.926938 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.934314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:41.934456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:41.937212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:41.937345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:41.937481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.937614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.937757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.937891 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.938033 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.938186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.938343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.938480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.938613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.938755 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.938887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.939020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.939155 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.939236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.939386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.939474 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.939662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.939747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.939931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.940016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.962531 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.962639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.964009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:41.964098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:41.964314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.964414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.969026 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:41.969114 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:41.969313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.969400 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.969595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.969681 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.976053 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:41.976142 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:41.976854 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.976945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.982806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:41.982904 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:41.983101 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.983186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.983393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.983522 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.983677 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.983775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.983951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.984035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.984226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.984310 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.984493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.984576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.984762 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:41.984843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.000587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.000865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.000953 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.001148 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.004665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.004973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.007883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.008178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.008264 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.008515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.012305 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.012686 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.015479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.015788 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.018699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.019002 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.019088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.019294 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.019381 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.019582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.019677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.019865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.019951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.020162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.020248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.020504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.036347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.036633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.036718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.036904 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.036989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.037167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.037254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.037502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.044574 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.044917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.046155 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.046507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.047675 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.047952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.048036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.048413 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.050712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.050998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.051124 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.051284 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.051435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.051573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.051681 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.051968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.052059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.052370 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.063571 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.063859 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.064031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.064148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.064319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.064420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.064608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.064806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.064889 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.065101 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.065271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.065383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.065569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.065855 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.081306 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.081719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.084235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.084661 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.087165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.087572 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.088601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.089041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.090066 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.090378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.090528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.090781 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.090960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.091212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.097184 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.097562 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.098613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.099063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.101527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.101923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.102030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.102236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.102405 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.102507 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.102700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.102987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.111340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.111340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.111536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.111626 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.111705 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.111705 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.118440 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81447) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.118724 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.118962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:51:42.119051 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.119250 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.119335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.119620 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.119729 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.119810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.119884 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.120067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.120156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.120345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.120446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.120615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.120701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.120894 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.120979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.121167 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.121260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.121428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.121559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.121696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.121782 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.121967 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81447) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.122052 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.122257 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.122532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:51:42.156868 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5589406039c0 pid: 81447 (default app) Sep 09 11:51:42.157418 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5589406039c0 pid: 81448 (default app) Sep 09 11:51:42.160295 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e62147-7ee6-49dc-aa2c-0c2442621f97 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:42.160574 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e62147-7ee6-49dc-aa2c-0c2442621f97 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:42.160847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e62147-7ee6-49dc-aa2c-0c2442621f97 None None] PATH_INFO: `/v3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:42.165636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1/1] 10.222.0.62 () {50 vars in 791 bytes} [Fri Sep 9 11:51:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:51:49.028602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9accd-8ed4-4733-9187-992c18c20c94 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:49.029120 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9accd-8ed4-4733-9187-992c18c20c94 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:49.029228 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9accd-8ed4-4733-9187-992c18c20c94 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:49.033501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1/2] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:51:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:51:49.038911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306a9a2-3599-4a84-bb67-8473400852a9 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:49.039430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306a9a2-3599-4a84-bb67-8473400852a9 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:49.039430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306a9a2-3599-4a84-bb67-8473400852a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:49.109359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: INFO dbcounter [None req-4306a9a2-3599-4a84-bb67-8473400852a9 None None] Registered counter for database keystone Sep 09 11:51:49.128015 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG oslo_db.sqlalchemy.engines [None req-4306a9a2-3599-4a84-bb67-8473400852a9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81447) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 11:51:49.134775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writer thread running {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 11:51:49.313819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG passlib.handlers.bcrypt [None req-4306a9a2-3599-4a84-bb67-8473400852a9 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81447) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 11:51:49.314147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG passlib.handlers.bcrypt [None req-4306a9a2-3599-4a84-bb67-8473400852a9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81447) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 11:51:49.373053 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-4306a9a2-3599-4a84-bb67-8473400852a9 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:51:49.374676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-4306a9a2-3599-4a84-bb67-8473400852a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:49.502223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 2/3] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:51:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 11:51:49.991173 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9accd-8ed4-4733-9187-992c18c20c94 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:50.061408 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: INFO dbcounter [None req-ecc9accd-8ed4-4733-9187-992c18c20c94 None None] Registered counter for database keystone Sep 09 11:51:50.082739 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG oslo_db.sqlalchemy.engines [None req-ecc9accd-8ed4-4733-9187-992c18c20c94 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81448) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 11:51:50.088982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writer thread running {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 11:51:50.217938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9accd-8ed4-4733-9187-992c18c20c94 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:50.222339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa315f-af74-4d6f-9dd0-a18a81a07ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:50.222971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa315f-af74-4d6f-9dd0-a18a81a07ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:50.223254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa315f-af74-4d6f-9dd0-a18a81a07ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:50.223528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa315f-af74-4d6f-9dd0-a18a81a07ef2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:50.232447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa315f-af74-4d6f-9dd0-a18a81a07ef2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:50.290940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ccfa315f-af74-4d6f-9dd0-a18a81a07ef2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 11:51:50.291221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa315f-af74-4d6f-9dd0-a18a81a07ef2 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:50.301174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 2/4] 10.222.0.62 () {58 vars in 1324 bytes} [Fri Sep 9 11:51:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:51:50.307464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306a9a2-3599-4a84-bb67-8473400852a9 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:50.324203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306a9a2-3599-4a84-bb67-8473400852a9 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:50.328608 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157f13f-505f-4e92-aa22-f775a44220ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:50.329733 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157f13f-505f-4e92-aa22-f775a44220ec admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:50.330015 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157f13f-505f-4e92-aa22-f775a44220ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:50.330318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157f13f-505f-4e92-aa22-f775a44220ec admin admin] PATH_INFO: `/v3/projects/fe7d1fa6c42548518e72832336a573e5` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:50.331898 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157f13f-505f-4e92-aa22-f775a44220ec admin admin] RBAC: Authorizing `identity:get_project(project_id=fe7d1fa6c42548518e72832336a573e5)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:50.384522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a157f13f-505f-4e92-aa22-f775a44220ec admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 11:51:50.384859 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157f13f-505f-4e92-aa22-f775a44220ec admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:50.389462 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 3/5] 10.222.0.62 () {56 vars in 1207 bytes} [Fri Sep 9 11:51:50 2022] GET /identity/v3/projects/fe7d1fa6c42548518e72832336a573e5 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:51:52.505053 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8f151-704e-4a2c-a3a9-1b01782e8a64 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:52.505753 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8f151-704e-4a2c-a3a9-1b01782e8a64 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:52.505753 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8f151-704e-4a2c-a3a9-1b01782e8a64 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:52.510271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 3/6] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:51:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:51:52.516318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d47a5-dfee-4174-be8e-6373e46f5eec None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:52.516799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d47a5-dfee-4174-be8e-6373e46f5eec None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:52.516896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d47a5-dfee-4174-be8e-6373e46f5eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:52.540374 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-717d47a5-dfee-4174-be8e-6373e46f5eec None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:51:52.541922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-717d47a5-dfee-4174-be8e-6373e46f5eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:52.549984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 4/7] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:51:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 11:51:52.818390 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8f151-704e-4a2c-a3a9-1b01782e8a64 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:52.831006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8f151-704e-4a2c-a3a9-1b01782e8a64 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:52.835222 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d708ea-7742-4984-8cca-9cbb92ae9818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:52.835812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d708ea-7742-4984-8cca-9cbb92ae9818 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:52.836088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d708ea-7742-4984-8cca-9cbb92ae9818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:52.836377 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d708ea-7742-4984-8cca-9cbb92ae9818 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:52.843967 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d708ea-7742-4984-8cca-9cbb92ae9818 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:52.845721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d708ea-7742-4984-8cca-9cbb92ae9818 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:52.854374 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 4/8] 10.222.0.62 () {58 vars in 1324 bytes} [Fri Sep 9 11:51:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:51:52.860506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d47a5-dfee-4174-be8e-6373e46f5eec None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:52.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d47a5-dfee-4174-be8e-6373e46f5eec None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:52.874581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b6602-6529-4e10-b308-b46455435a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:52.875216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b6602-6529-4e10-b308-b46455435a5d admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:52.875495 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b6602-6529-4e10-b308-b46455435a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:52.875768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b6602-6529-4e10-b308-b46455435a5d admin admin] PATH_INFO: `/v3/users/5c92b48d12534a97833b0661f5d186ee` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:52.886005 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b6602-6529-4e10-b308-b46455435a5d admin admin] RBAC: Authorizing `identity:get_user(user_id=5c92b48d12534a97833b0661f5d186ee)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:52.888079 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b6602-6529-4e10-b308-b46455435a5d admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:52.889818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 5/9] 10.222.0.62 () {56 vars in 1198 bytes} [Fri Sep 9 11:51:52 2022] GET /identity/v3/users/5c92b48d12534a97833b0661f5d186ee => generated 247 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:51:55.120267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bac183-b72d-423d-846d-1e2badb75a84 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:55.120812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bac183-b72d-423d-846d-1e2badb75a84 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:55.120812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bac183-b72d-423d-846d-1e2badb75a84 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:55.122248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 5/10] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:51:55.127537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704aa68-fa84-461e-a50d-ea5bb436efcf None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:55.127998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704aa68-fa84-461e-a50d-ea5bb436efcf None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:55.128215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704aa68-fa84-461e-a50d-ea5bb436efcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:55.146709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f704aa68-fa84-461e-a50d-ea5bb436efcf None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:51:55.147895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f704aa68-fa84-461e-a50d-ea5bb436efcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:55.157673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0361fcd-864a-484e-8615-6cb1bbcc1601 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:55.157955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0361fcd-864a-484e-8615-6cb1bbcc1601 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:55.158261 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0361fcd-864a-484e-8615-6cb1bbcc1601 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:55.159583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 6/11] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:51:55.164773 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748180-261e-45c9-ba26-9f5cd26b9552 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:55.165049 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748180-261e-45c9-ba26-9f5cd26b9552 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:55.165313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748180-261e-45c9-ba26-9f5cd26b9552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:55.208006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG passlib.handlers.bcrypt [None req-96748180-261e-45c9-ba26-9f5cd26b9552 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81448) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 11:51:55.208390 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG passlib.handlers.bcrypt [None req-96748180-261e-45c9-ba26-9f5cd26b9552 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81448) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 11:51:55.233611 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 6/12] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:51:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:51:55.259957 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-96748180-261e-45c9-ba26-9f5cd26b9552 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:51:55.261556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-96748180-261e-45c9-ba26-9f5cd26b9552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:55.359825 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 7/13] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:51:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:51:55.501401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704aa68-fa84-461e-a50d-ea5bb436efcf None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:55.526084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:51:55.611524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704aa68-fa84-461e-a50d-ea5bb436efcf None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:55.634969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96748180-261e-45c9-ba26-9f5cd26b9552 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:55.645675 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a268d97-cea9-4743-8f5e-078decdeb41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:55.646341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a268d97-cea9-4743-8f5e-078decdeb41b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:55.646628 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a268d97-cea9-4743-8f5e-078decdeb41b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:55.646913 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a268d97-cea9-4743-8f5e-078decdeb41b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:55.658217 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a268d97-cea9-4743-8f5e-078decdeb41b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-7a268d97-cea9-4743-8f5e-078decdeb41b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:51:55.667140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:51:55.668349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:51:55.669351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:51:55.669351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:51:55.669351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:51:55.669351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:51:55.669351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:51:55.669351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:51:55.669351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:51:55.669351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:51:55.669351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:51:55.669351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:51:55.669351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:51:55.669351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:51:55.669351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 7/14] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:51:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:51:55.674016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a268d97-cea9-4743-8f5e-078decdeb41b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:55.736824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96748180-261e-45c9-ba26-9f5cd26b9552 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:55.739098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:51:55.757242 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:51:55.776589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a268d97-cea9-4743-8f5e-078decdeb41b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:55.780787 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc5b97-9bb7-412c-a2f8-6e3f6355a046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:55.781667 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc5b97-9bb7-412c-a2f8-6e3f6355a046 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:55.781945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc5b97-9bb7-412c-a2f8-6e3f6355a046 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:55.782247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc5b97-9bb7-412c-a2f8-6e3f6355a046 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:55.815519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fad3e4-79b8-42bc-ac03-d2069dd3e2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:55.816142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fad3e4-79b8-42bc-ac03-d2069dd3e2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:55.816418 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fad3e4-79b8-42bc-ac03-d2069dd3e2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:55.816687 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fad3e4-79b8-42bc-ac03-d2069dd3e2e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:55.817372 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fad3e4-79b8-42bc-ac03-d2069dd3e2e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:51:55.817669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fad3e4-79b8-42bc-ac03-d2069dd3e2e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:55.819099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fad3e4-79b8-42bc-ac03-d2069dd3e2e6 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:55.836821 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 8/15] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:51:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:51:55.844495 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fad3e4-79b8-42bc-ac03-d2069dd3e2e6 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:55.865456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc5b97-9bb7-412c-a2f8-6e3f6355a046 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:55.867127 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc5b97-9bb7-412c-a2f8-6e3f6355a046 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:55.952314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fad3e4-79b8-42bc-ac03-d2069dd3e2e6 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:55.970713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 8/16] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:51:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:51:55.976368 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc5b97-9bb7-412c-a2f8-6e3f6355a046 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:55.979640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:51:55.989680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4dd49a-dc1b-42a6-be9e-96e7cc736edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:55.990353 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dd49a-dc1b-42a6-be9e-96e7cc736edd None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:55.990637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dd49a-dc1b-42a6-be9e-96e7cc736edd None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:55.990911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dd49a-dc1b-42a6-be9e-96e7cc736edd None admin] PATH_INFO: `/v3/users/5c92b48d12534a97833b0661f5d186ee` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:55.992062 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dd49a-dc1b-42a6-be9e-96e7cc736edd None admin] RBAC: Authorizing `identity:get_user(user_id=5c92b48d12534a97833b0661f5d186ee)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:55.994143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dd49a-dc1b-42a6-be9e-96e7cc736edd None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:55.995750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 9/17] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:51:55 2022] GET /identity/v3/users/5c92b48d12534a97833b0661f5d186ee => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:51:56.001037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4dd49a-dc1b-42a6-be9e-96e7cc736edd None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:56.016094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:51:56.075933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc5b97-9bb7-412c-a2f8-6e3f6355a046 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:56.100338 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4dd49a-dc1b-42a6-be9e-96e7cc736edd None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:56.109591 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fcba1-fcf8-4e03-9813-f6e7442ac525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:56.110456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fcba1-fcf8-4e03-9813-f6e7442ac525 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:56.110733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fcba1-fcf8-4e03-9813-f6e7442ac525 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:56.111001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fcba1-fcf8-4e03-9813-f6e7442ac525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:56.130147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563fcba1-fcf8-4e03-9813-f6e7442ac525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:56.131627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563fcba1-fcf8-4e03-9813-f6e7442ac525 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:56.136675 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb846363-cae8-48e4-a176-673a1c4fe2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:56.137310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb846363-cae8-48e4-a176-673a1c4fe2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:56.137587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb846363-cae8-48e4-a176-673a1c4fe2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:56.137855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb846363-cae8-48e4-a176-673a1c4fe2b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:56.139019 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb846363-cae8-48e4-a176-673a1c4fe2b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:56.140455 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb846363-cae8-48e4-a176-673a1c4fe2b0 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:56.142324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 10/18] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:51:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-563fcba1-fcf8-4e03-9813-f6e7442ac525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:51:56.150394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:51:56.151497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:51:56.152303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:51:56.152303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:51:56.152303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:51:56.152476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb846363-cae8-48e4-a176-673a1c4fe2b0 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:56.152548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 9/19] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:51:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:51:56.157120 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fcba1-fcf8-4e03-9813-f6e7442ac525 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:56.215848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:51:56.239463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:51:56.259988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb846363-cae8-48e4-a176-673a1c4fe2b0 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:56.261439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fcba1-fcf8-4e03-9813-f6e7442ac525 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:56.295524 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f1c65-f8b4-4f4b-ba5b-961ad541b59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:56.296063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f1c65-f8b4-4f4b-ba5b-961ad541b59f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:56.296346 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f1c65-f8b4-4f4b-ba5b-961ad541b59f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:56.296620 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f1c65-f8b4-4f4b-ba5b-961ad541b59f None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:56.296922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f26d5-61ee-4c75-8df2-b7664e8934a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:56.297268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f1c65-f8b4-4f4b-ba5b-961ad541b59f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:51:56.297522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f26d5-61ee-4c75-8df2-b7664e8934a1 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:56.297624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f1c65-f8b4-4f4b-ba5b-961ad541b59f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:56.297796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f26d5-61ee-4c75-8df2-b7664e8934a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:56.298063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f26d5-61ee-4c75-8df2-b7664e8934a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:56.298808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f26d5-61ee-4c75-8df2-b7664e8934a1 None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.domain.id=default, user.id=5c92b48d12534a97833b0661f5d186ee) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:51:56.298958 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f1c65-f8b4-4f4b-ba5b-961ad541b59f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:56.299111 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f26d5-61ee-4c75-8df2-b7664e8934a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:56.300571 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f26d5-61ee-4c75-8df2-b7664e8934a1 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:56.307747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 11/20] 10.222.0.62 () {56 vars in 1322 bytes} [Fri Sep 9 11:51:56 2022] GET /identity/v3/role_assignments?user.id=5c92b48d12534a97833b0661f5d186ee&scope.domain.id=default&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:51:56.317497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 10/21] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:51:58.796416 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7921a-81e7-41df-b23a-76eb2c37361e None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:58.796960 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7921a-81e7-41df-b23a-76eb2c37361e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:58.796960 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7921a-81e7-41df-b23a-76eb2c37361e None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:58.798519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 12/22] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:51:58.804125 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54c8df-3941-4e15-a8ab-968ea370c643 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:58.805544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54c8df-3941-4e15-a8ab-968ea370c643 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:58.806020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54c8df-3941-4e15-a8ab-968ea370c643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:58.826047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4c54c8df-3941-4e15-a8ab-968ea370c643 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:51:58.827625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4c54c8df-3941-4e15-a8ab-968ea370c643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:58.904992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 11/23] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:51:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:51:58.996267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03e52b-3d98-4dd2-b2f8-9e0def2724b8 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:58.996907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03e52b-3d98-4dd2-b2f8-9e0def2724b8 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:58.997398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03e52b-3d98-4dd2-b2f8-9e0def2724b8 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:58.998978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 13/24] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:51:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:51:59.004527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80afcf-ecb3-41ff-bce4-9641e349806d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:59.004815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80afcf-ecb3-41ff-bce4-9641e349806d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:59.005095 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80afcf-ecb3-41ff-bce4-9641e349806d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:59.030938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-3f80afcf-ecb3-41ff-bce4-9641e349806d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:51:59.032162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3f80afcf-ecb3-41ff-bce4-9641e349806d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:59.081776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:51:59.108127 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 12/25] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:51:59.361131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03e52b-3d98-4dd2-b2f8-9e0def2724b8 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:59.411982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80afcf-ecb3-41ff-bce4-9641e349806d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:59.531581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80afcf-ecb3-41ff-bce4-9641e349806d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:59.562940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03e52b-3d98-4dd2-b2f8-9e0def2724b8 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:59.576381 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbf699-c732-4cf8-8616-4dbbd7530dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:59.577432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbf699-c732-4cf8-8616-4dbbd7530dfd None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:59.577974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbf699-c732-4cf8-8616-4dbbd7530dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:59.578545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbf699-c732-4cf8-8616-4dbbd7530dfd None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:59.579535 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbf699-c732-4cf8-8616-4dbbd7530dfd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:51:59.581983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbf699-c732-4cf8-8616-4dbbd7530dfd None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:51:59.650985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae0b23-e18e-4f36-8c48-94135cec4c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:51:59.651572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae0b23-e18e-4f36-8c48-94135cec4c54 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:59.651857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae0b23-e18e-4f36-8c48-94135cec4c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:59.652129 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae0b23-e18e-4f36-8c48-94135cec4c54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:59.679317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae0b23-e18e-4f36-8c48-94135cec4c54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-94ae0b23-e18e-4f36-8c48-94135cec4c54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:51:59.685542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:51:59.686709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:51:59.687489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:51:59.687489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:51:59.687489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:51:59.687489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:51:59.687489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:51:59.687489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:51:59.687489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:51:59.687489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:51:59.687489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:51:59.687489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:51:59.687489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:51:59.687489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:51:59.687489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 14/26] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:51:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:51:59.688150 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9309c5-2a31-46ba-bc11-b5173010ce0c None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:59.689067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9309c5-2a31-46ba-bc11-b5173010ce0c None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:59.689067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9309c5-2a31-46ba-bc11-b5173010ce0c None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:59.691460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 15/27] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:51:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:51:59.693248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76546ad-2992-4d57-bab3-84b5c6d64cd4 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:59.693621 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76546ad-2992-4d57-bab3-84b5c6d64cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:59.693903 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76546ad-2992-4d57-bab3-84b5c6d64cd4 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:59.695197 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 16/28] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:51:59.696462 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76546ad-2992-4d57-bab3-84b5c6d64cd4 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:51:59.708271 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:51:59.737751 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 13/29] 10.222.0.62 () {60 vars in 1131 bytes} [Fri Sep 9 11:51:59 2022] POST /identity/v3/roles => generated 219 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:51:59.739516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884b4cf-0aaf-4a4a-9e66-10f33a16807c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:59.739813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884b4cf-0aaf-4a4a-9e66-10f33a16807c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:59.740084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884b4cf-0aaf-4a4a-9e66-10f33a16807c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:59.786323 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9884b4cf-0aaf-4a4a-9e66-10f33a16807c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:51:59.788903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9884b4cf-0aaf-4a4a-9e66-10f33a16807c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:51:59.859905 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:51:59.921717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76546ad-2992-4d57-bab3-84b5c6d64cd4 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:51:59.947457 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 14/30] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:51:59.949027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d08d9-20af-4979-afb0-469f299f4ba8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:51:59.949606 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d08d9-20af-4979-afb0-469f299f4ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:51:59.958655 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d08d9-20af-4979-afb0-469f299f4ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:51:59.979313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ad4d08d9-20af-4979-afb0-469f299f4ba8 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:51:59.980874 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ad4d08d9-20af-4979-afb0-469f299f4ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:00.017915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b42c3-6407-4a37-aae0-1d498f9494ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:00.018936 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b42c3-6407-4a37-aae0-1d498f9494ff None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:00.019521 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b42c3-6407-4a37-aae0-1d498f9494ff None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:00.019988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b42c3-6407-4a37-aae0-1d498f9494ff None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:00.020909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b42c3-6407-4a37-aae0-1d498f9494ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:00.021406 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b42c3-6407-4a37-aae0-1d498f9494ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:00.023044 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b42c3-6407-4a37-aae0-1d498f9494ff None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:00.041959 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 17/31] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:51:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:00.052668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b42c3-6407-4a37-aae0-1d498f9494ff None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:00.063777 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 15/32] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:00.167114 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:00.216667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b42c3-6407-4a37-aae0-1d498f9494ff None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:00.263566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bae19-8e99-466d-aa42-9b0093d719f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:00.264198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bae19-8e99-466d-aa42-9b0093d719f9 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:00.264473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bae19-8e99-466d-aa42-9b0093d719f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:00.264740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bae19-8e99-466d-aa42-9b0093d719f9 None admin] PATH_INFO: `/v3/users/5c92b48d12534a97833b0661f5d186ee` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:00.271064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1bae19-8e99-466d-aa42-9b0093d719f9 None admin] RBAC: Authorizing `identity:get_user(user_id=5c92b48d12534a97833b0661f5d186ee)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:00.272586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1bae19-8e99-466d-aa42-9b0093d719f9 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:00.274698 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 18/33] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:00 2022] GET /identity/v3/users/5c92b48d12534a97833b0661f5d186ee => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:00.281492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d08d9-20af-4979-afb0-469f299f4ba8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:00.365482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bae19-8e99-466d-aa42-9b0093d719f9 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:00.422177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:00.429279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d08d9-20af-4979-afb0-469f299f4ba8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:00.466282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252659b-4eb9-4ff4-bfeb-e408ffd036ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:00.466977 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252659b-4eb9-4ff4-bfeb-e408ffd036ce None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:00.467208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252659b-4eb9-4ff4-bfeb-e408ffd036ce None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:00.467483 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252659b-4eb9-4ff4-bfeb-e408ffd036ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:00.468961 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252659b-4eb9-4ff4-bfeb-e408ffd036ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:00.470566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252659b-4eb9-4ff4-bfeb-e408ffd036ce None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:00.472815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 16/34] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:52:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:00.473286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2591c-b182-4b42-9b18-5292f237d8fe None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:00.473564 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2591c-b182-4b42-9b18-5292f237d8fe None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:00.473833 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2591c-b182-4b42-9b18-5292f237d8fe None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:00.478231 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 17/35] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:00.478231 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2591c-b182-4b42-9b18-5292f237d8fe None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:00.511144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bae19-8e99-466d-aa42-9b0093d719f9 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:00.549373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ba4bc-d6d5-448e-b143-5364b6a44f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:00.550007 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ba4bc-d6d5-448e-b143-5364b6a44f20 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:00.550314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ba4bc-d6d5-448e-b143-5364b6a44f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:00.550590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ba4bc-d6d5-448e-b143-5364b6a44f20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:00.552889 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:00.564311 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ba4bc-d6d5-448e-b143-5364b6a44f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:00.565779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ba4bc-d6d5-448e-b143-5364b6a44f20 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-409ba4bc-d6d5-448e-b143-5364b6a44f20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:52:00.582324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:00.583236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:52:00.584035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:52:00.584035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:52:00.584035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:00.584035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 19/36] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:52:00.584246 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ba4bc-d6d5-448e-b143-5364b6a44f20 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:00.595040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2591c-b182-4b42-9b18-5292f237d8fe None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:00.632785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9a14f-652b-43b7-a75b-5c6402e43ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:00.633460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9a14f-652b-43b7-a75b-5c6402e43ad0 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:00.633745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9a14f-652b-43b7-a75b-5c6402e43ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:00.634021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9a14f-652b-43b7-a75b-5c6402e43ad0 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:00.634734 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9a14f-652b-43b7-a75b-5c6402e43ad0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:00.636146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9a14f-652b-43b7-a75b-5c6402e43ad0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:00.659298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 18/37] 10.222.0.62 () {60 vars in 1131 bytes} [Fri Sep 9 11:52:00 2022] POST /identity/v3/roles => generated 225 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:52:00.662301 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8036cd-d003-463f-bd86-822173a90862 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:00.663162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8036cd-d003-463f-bd86-822173a90862 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:00.663706 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8036cd-d003-463f-bd86-822173a90862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:00.698042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-3a8036cd-d003-463f-bd86-822173a90862 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:00.700898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3a8036cd-d003-463f-bd86-822173a90862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:00.755419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:00.763808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ba4bc-d6d5-448e-b143-5364b6a44f20 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:00.810347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6048c-2132-4e3c-a4b5-d852026056f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:00.811028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6048c-2132-4e3c-a4b5-d852026056f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:00.811308 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6048c-2132-4e3c-a4b5-d852026056f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:00.811582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6048c-2132-4e3c-a4b5-d852026056f9 None admin] PATH_INFO: `/v3/domains/default/users/5c92b48d12534a97833b0661f5d186ee/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:00.812941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 19/38] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:00.813639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8036cd-d003-463f-bd86-822173a90862 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:00.819270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6048c-2132-4e3c-a4b5-d852026056f9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5c92b48d12534a97833b0661f5d186ee, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:00.821387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6048c-2132-4e3c-a4b5-d852026056f9 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:00.941808 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8036cd-d003-463f-bd86-822173a90862 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:00.946510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 20/39] 10.222.0.62 () {58 vars in 1361 bytes} [Fri Sep 9 11:52:00 2022] PUT /identity/v3/domains/default/users/5c92b48d12534a97833b0661f5d186ee/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:52:00.948101 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd0b20-d3e2-4ed5-b0ff-b30e571ee02d None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:00.950738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd0b20-d3e2-4ed5-b0ff-b30e571ee02d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:00.950738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd0b20-d3e2-4ed5-b0ff-b30e571ee02d None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:00.953693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 21/40] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:00.959207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954fdb6-924a-45c6-a75d-5014aa1b92b6 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:00.959590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954fdb6-924a-45c6-a75d-5014aa1b92b6 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:00.959943 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954fdb6-924a-45c6-a75d-5014aa1b92b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:00.980517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-b954fdb6-924a-45c6-a75d-5014aa1b92b6 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:00.982521 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b954fdb6-924a-45c6-a75d-5014aa1b92b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:00.987723 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1a47a-a4d2-4adc-bde0-c2afc0a47d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:00.988647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1a47a-a4d2-4adc-bde0-c2afc0a47d10 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:00.989134 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1a47a-a4d2-4adc-bde0-c2afc0a47d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:00.989652 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1a47a-a4d2-4adc-bde0-c2afc0a47d10 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:00.990546 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1a47a-a4d2-4adc-bde0-c2afc0a47d10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:00.991060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1a47a-a4d2-4adc-bde0-c2afc0a47d10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:00.993300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1a47a-a4d2-4adc-bde0-c2afc0a47d10 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:01.006729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 20/41] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:01.013075 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1a47a-a4d2-4adc-bde0-c2afc0a47d10 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:01.035148 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:01.041594 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:01.083011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 22/42] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:01.099876 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954fdb6-924a-45c6-a75d-5014aa1b92b6 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:01.168026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1a47a-a4d2-4adc-bde0-c2afc0a47d10 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:01.212066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edea49-ecf9-4be4-938c-9584734f97a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:01.212997 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edea49-ecf9-4be4-938c-9584734f97a4 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:01.213278 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edea49-ecf9-4be4-938c-9584734f97a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:01.213548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edea49-ecf9-4be4-938c-9584734f97a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:01.214494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1edea49-ecf9-4be4-938c-9584734f97a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:01.216350 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1edea49-ecf9-4be4-938c-9584734f97a4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:01.231476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954fdb6-924a-45c6-a75d-5014aa1b92b6 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:01.260437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 21/43] 10.222.0.62 () {60 vars in 1141 bytes} [Fri Sep 9 11:52:01 2022] POST /identity/v3/projects => generated 301 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:52:01.281254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bdde8-1adc-42c3-a467-5f6f10263ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:01.281891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bdde8-1adc-42c3-a467-5f6f10263ea4 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:01.282207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bdde8-1adc-42c3-a467-5f6f10263ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:01.282481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bdde8-1adc-42c3-a467-5f6f10263ea4 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:01.283309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6bdde8-1adc-42c3-a467-5f6f10263ea4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:01.285583 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6bdde8-1adc-42c3-a467-5f6f10263ea4 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:01.306947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 23/44] 10.222.0.62 () {60 vars in 1131 bytes} [Fri Sep 9 11:52:01 2022] POST /identity/v3/roles => generated 223 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:52:01.363724 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edea49-ecf9-4be4-938c-9584734f97a4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:01.452480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:01.494091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edea49-ecf9-4be4-938c-9584734f97a4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:01.532256 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884fdc9-6e77-43a6-9341-01b5bd314556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:01.532816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884fdc9-6e77-43a6-9341-01b5bd314556 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:01.533093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884fdc9-6e77-43a6-9341-01b5bd314556 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:01.533367 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884fdc9-6e77-43a6-9341-01b5bd314556 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:01.534663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884fdc9-6e77-43a6-9341-01b5bd314556 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:01.536212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884fdc9-6e77-43a6-9341-01b5bd314556 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:01.537978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 22/45] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:52:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:01.544377 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bdde8-1adc-42c3-a467-5f6f10263ea4 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:01.583312 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:01.758870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bdde8-1adc-42c3-a467-5f6f10263ea4 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:01.794064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054d796-1c22-4756-b005-ac59ceb681f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:01.794622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054d796-1c22-4756-b005-ac59ceb681f2 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:01.794901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054d796-1c22-4756-b005-ac59ceb681f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:01.795171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054d796-1c22-4756-b005-ac59ceb681f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:01.795828 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054d796-1c22-4756-b005-ac59ceb681f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:01.797281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054d796-1c22-4756-b005-ac59ceb681f2 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:01.826236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 24/46] 10.222.0.62 () {60 vars in 1141 bytes} [Fri Sep 9 11:52:01 2022] POST /identity/v3/projects => generated 312 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:52:03.216122 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd345e-0c86-49f9-b203-fef8c109dcdb None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:03.216668 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd345e-0c86-49f9-b203-fef8c109dcdb None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:03.216668 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd345e-0c86-49f9-b203-fef8c109dcdb None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:03.218027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 23/47] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:03.223672 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72017c-b8ee-4c7e-bff0-a16b6ebab81d None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:03.224086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72017c-b8ee-4c7e-bff0-a16b6ebab81d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:03.224278 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72017c-b8ee-4c7e-bff0-a16b6ebab81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:03.243080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-ef72017c-b8ee-4c7e-bff0-a16b6ebab81d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:03.244266 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ef72017c-b8ee-4c7e-bff0-a16b6ebab81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:03.310983 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 25/48] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:03.581455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd345e-0c86-49f9-b203-fef8c109dcdb None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:03.683524 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd345e-0c86-49f9-b203-fef8c109dcdb None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:03.718315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67ee01-1aa1-42af-a547-cc76ede6ef22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:03.718951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67ee01-1aa1-42af-a547-cc76ede6ef22 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:03.719232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67ee01-1aa1-42af-a547-cc76ede6ef22 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:03.719508 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67ee01-1aa1-42af-a547-cc76ede6ef22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:03.721461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:03.731738 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67ee01-1aa1-42af-a547-cc76ede6ef22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-8d67ee01-1aa1-42af-a547-cc76ede6ef22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:03.841710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:03.842675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:03.843426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:03.843426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:03.843426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:03.843426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:03.843426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:03.843426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:03.843426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:03.843426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:03.843426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:03.843426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:03.843426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:03.843426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:03.843426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 24/49] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:52:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:03.850102 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72017c-b8ee-4c7e-bff0-a16b6ebab81d None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:03.870907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:03.956120 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72017c-b8ee-4c7e-bff0-a16b6ebab81d None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:03.980656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98956f07-4271-4add-bb1a-2104a6a65d24 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:03.980942 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98956f07-4271-4add-bb1a-2104a6a65d24 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:03.981216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98956f07-4271-4add-bb1a-2104a6a65d24 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:03.982623 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 25/50] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:52:03.987716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3dd60d-eeaa-46c1-8e12-b27263d728e8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:03.987995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3dd60d-eeaa-46c1-8e12-b27263d728e8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:03.988267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3dd60d-eeaa-46c1-8e12-b27263d728e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:03.990015 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9665a9a-8d26-4fa9-b5d4-353846711845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:03.990654 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9665a9a-8d26-4fa9-b5d4-353846711845 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:03.990940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9665a9a-8d26-4fa9-b5d4-353846711845 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:03.991307 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9665a9a-8d26-4fa9-b5d4-353846711845 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:03.991891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9665a9a-8d26-4fa9-b5d4-353846711845 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:03.992196 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9665a9a-8d26-4fa9-b5d4-353846711845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:03.994145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9665a9a-8d26-4fa9-b5d4-353846711845 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:04.003470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 26/51] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:04.006682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9d3dd60d-eeaa-46c1-8e12-b27263d728e8 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:04.007895 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9d3dd60d-eeaa-46c1-8e12-b27263d728e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:04.009847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9665a9a-8d26-4fa9-b5d4-353846711845 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:04.076789 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 26/52] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:04.090497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:04.110818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9665a9a-8d26-4fa9-b5d4-353846711845 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:04.147398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e481b-f64c-4b89-ae4c-cd592ffbc500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:04.148029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e481b-f64c-4b89-ae4c-cd592ffbc500 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:04.148310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e481b-f64c-4b89-ae4c-cd592ffbc500 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:04.148622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e481b-f64c-4b89-ae4c-cd592ffbc500 None admin] PATH_INFO: `/v3/users/5c92b48d12534a97833b0661f5d186ee` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:04.150378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e481b-f64c-4b89-ae4c-cd592ffbc500 None admin] RBAC: Authorizing `identity:get_user(user_id=5c92b48d12534a97833b0661f5d186ee)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:04.151833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e481b-f64c-4b89-ae4c-cd592ffbc500 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:04.153438 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 27/53] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:04 2022] GET /identity/v3/users/5c92b48d12534a97833b0661f5d186ee => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:04.158936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3dd60d-eeaa-46c1-8e12-b27263d728e8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:04.257699 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3dd60d-eeaa-46c1-8e12-b27263d728e8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:04.281002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:04.291947 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5facd-605b-4e8d-8b35-197f4e91393a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:04.292581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5facd-605b-4e8d-8b35-197f4e91393a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:04.292859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5facd-605b-4e8d-8b35-197f4e91393a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:04.293133 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5facd-605b-4e8d-8b35-197f4e91393a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:04.294312 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5facd-605b-4e8d-8b35-197f4e91393a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:04.295759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5facd-605b-4e8d-8b35-197f4e91393a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:04.297356 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 27/54] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:52:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:04.303142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e481b-f64c-4b89-ae4c-cd592ffbc500 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:04.346345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5facd-605b-4e8d-8b35-197f4e91393a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:04.407969 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e481b-f64c-4b89-ae4c-cd592ffbc500 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:04.440533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf8531-724d-4c6b-8137-a280e4e583fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:04.441148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf8531-724d-4c6b-8137-a280e4e583fc None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:04.441430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf8531-724d-4c6b-8137-a280e4e583fc None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:04.441713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf8531-724d-4c6b-8137-a280e4e583fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:04.442473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf8531-724d-4c6b-8137-a280e4e583fc None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.domain.id=default, user.id=5c92b48d12534a97833b0661f5d186ee) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:04.442780 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf8531-724d-4c6b-8137-a280e4e583fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:04.444240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf8531-724d-4c6b-8137-a280e4e583fc None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:04.447702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5facd-605b-4e8d-8b35-197f4e91393a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:04.450550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 28/55] 10.222.0.62 () {56 vars in 1322 bytes} [Fri Sep 9 11:52:04 2022] GET /identity/v3/role_assignments?user.id=5c92b48d12534a97833b0661f5d186ee&scope.domain.id=default&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:04.483986 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23923071-70b1-4c31-90ab-634591de7855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:04.484638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23923071-70b1-4c31-90ab-634591de7855 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:04.484915 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23923071-70b1-4c31-90ab-634591de7855 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:04.485185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23923071-70b1-4c31-90ab-634591de7855 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:04.486401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23923071-70b1-4c31-90ab-634591de7855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:04.487870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23923071-70b1-4c31-90ab-634591de7855 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:04.489517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 28/56] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:52:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:04.495881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf8531-724d-4c6b-8137-a280e4e583fc None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:04.514751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:04.607161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf8531-724d-4c6b-8137-a280e4e583fc None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:04.644206 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e09fb-82c2-4ab7-94d2-33e2386adce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:04.644817 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e09fb-82c2-4ab7-94d2-33e2386adce2 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:04.645714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e09fb-82c2-4ab7-94d2-33e2386adce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:04.645714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e09fb-82c2-4ab7-94d2-33e2386adce2 None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:04.646005 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e09fb-82c2-4ab7-94d2-33e2386adce2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:04.647492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e09fb-82c2-4ab7-94d2-33e2386adce2 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:04.674333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 29/57] 10.222.0.62 () {60 vars in 1140 bytes} [Fri Sep 9 11:52:04 2022] POST /identity/v3/projects => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:52:06.843914 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694020f4-d17d-445e-af7a-2cac547db299 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:06.844441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694020f4-d17d-445e-af7a-2cac547db299 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:06.844441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694020f4-d17d-445e-af7a-2cac547db299 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:06.845788 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 29/58] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:06.851149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd4912-44b5-4258-875d-18aad822cfa7 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:06.851528 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd4912-44b5-4258-875d-18aad822cfa7 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:06.851709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd4912-44b5-4258-875d-18aad822cfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:06.870235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-dfbd4912-44b5-4258-875d-18aad822cfa7 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:06.871444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-dfbd4912-44b5-4258-875d-18aad822cfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:06.928949 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:06.939093 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 30/59] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:07.217269 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694020f4-d17d-445e-af7a-2cac547db299 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:07.277697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:07.318670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694020f4-d17d-445e-af7a-2cac547db299 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:07.350747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4ddd2-7ede-4dce-af07-c175e038939c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:07.351367 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4ddd2-7ede-4dce-af07-c175e038939c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:07.351647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4ddd2-7ede-4dce-af07-c175e038939c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:07.352435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4ddd2-7ede-4dce-af07-c175e038939c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:07.353534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4ddd2-7ede-4dce-af07-c175e038939c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:07.354975 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4ddd2-7ede-4dce-af07-c175e038939c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:07.356518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 30/60] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:52:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:07.362376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd4912-44b5-4258-875d-18aad822cfa7 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:07.464549 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd4912-44b5-4258-875d-18aad822cfa7 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:07.499264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a01968-ccdc-4c43-b4f8-05e334bf8785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:07.499794 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a01968-ccdc-4c43-b4f8-05e334bf8785 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:07.500078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a01968-ccdc-4c43-b4f8-05e334bf8785 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:07.500354 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a01968-ccdc-4c43-b4f8-05e334bf8785 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:07.501024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a01968-ccdc-4c43-b4f8-05e334bf8785 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:07.502495 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a01968-ccdc-4c43-b4f8-05e334bf8785 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:07.528536 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 31/61] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:52:07 2022] POST /identity/v3/users => generated 275 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:52:10.693784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4a435-5e6a-4f40-9636-ec128068b87c None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:10.695335 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4a435-5e6a-4f40-9636-ec128068b87c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:10.696141 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4a435-5e6a-4f40-9636-ec128068b87c None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:10.699608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 31/62] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:10.706313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad4aa6-7d49-48a5-b2ed-ce7eafd52ba9 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:10.707672 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad4aa6-7d49-48a5-b2ed-ce7eafd52ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:10.708432 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad4aa6-7d49-48a5-b2ed-ce7eafd52ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:10.751819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-b2ad4aa6-7d49-48a5-b2ed-ce7eafd52ba9 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:10.753867 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b2ad4aa6-7d49-48a5-b2ed-ce7eafd52ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:10.871041 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:10.889637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 32/63] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:11.111846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609985cf-7992-431a-a206-f84b2b58f952 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:11.112873 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609985cf-7992-431a-a206-f84b2b58f952 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:11.113660 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609985cf-7992-431a-a206-f84b2b58f952 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:11.116849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 32/64] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:11.123237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028c6d4-1d7d-420f-a17a-a38c79f46b15 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:11.123843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028c6d4-1d7d-420f-a17a-a38c79f46b15 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:11.124320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028c6d4-1d7d-420f-a17a-a38c79f46b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:11.154482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-b028c6d4-1d7d-420f-a17a-a38c79f46b15 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:11.156046 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b028c6d4-1d7d-420f-a17a-a38c79f46b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:11.252376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 33/65] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:11.282807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609985cf-7992-431a-a206-f84b2b58f952 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:11.501939 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609985cf-7992-431a-a206-f84b2b58f952 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:11.542811 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:11.551058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87ef84-6e02-4430-8cd1-06c7c96c19b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:11.551750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87ef84-6e02-4430-8cd1-06c7c96c19b1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:11.552248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87ef84-6e02-4430-8cd1-06c7c96c19b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:11.552522 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87ef84-6e02-4430-8cd1-06c7c96c19b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:11.565839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87ef84-6e02-4430-8cd1-06c7c96c19b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-ef87ef84-6e02-4430-8cd1-06c7c96c19b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:11.575556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:11.576574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:11.577337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:11.577337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:11.577337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:11.577337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:11.577337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:11.577337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:11.577337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:11.577337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:11.577337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:11.577337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:11.577337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:11.577337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:11.577337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 33/66] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:11.583263 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028c6d4-1d7d-420f-a17a-a38c79f46b15 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:11.606775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87ef84-6e02-4430-8cd1-06c7c96c19b1 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:11.812266 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028c6d4-1d7d-420f-a17a-a38c79f46b15 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:11.826236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87ef84-6e02-4430-8cd1-06c7c96c19b1 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:11.835074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:11.868056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d56f4-9104-42bf-b4d9-df0bcbe41ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:11.869533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d56f4-9104-42bf-b4d9-df0bcbe41ffc None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:11.869825 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d56f4-9104-42bf-b4d9-df0bcbe41ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:11.870096 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d56f4-9104-42bf-b4d9-df0bcbe41ffc None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:11.871043 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d56f4-9104-42bf-b4d9-df0bcbe41ffc None admin] RBAC: Adding query filter params (name=member) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:11.871366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d56f4-9104-42bf-b4d9-df0bcbe41ffc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:11.872991 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d56f4-9104-42bf-b4d9-df0bcbe41ffc None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:11.875618 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93faf623-1a4b-4ce9-8a9f-1151c8597100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:11.876271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93faf623-1a4b-4ce9-8a9f-1151c8597100 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:11.876553 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93faf623-1a4b-4ce9-8a9f-1151c8597100 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:11.876828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93faf623-1a4b-4ce9-8a9f-1151c8597100 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:11.882285 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 34/67] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:11.883587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25943b8c-0aaa-46d3-9315-b7949069b700 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:11.883871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25943b8c-0aaa-46d3-9315-b7949069b700 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:11.884173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25943b8c-0aaa-46d3-9315-b7949069b700 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:11.885948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 35/68] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:11.887140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6e0c1-6616-4287-af33-6ad7dccf49a4 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:11.887546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6e0c1-6616-4287-af33-6ad7dccf49a4 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:11.887962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6e0c1-6616-4287-af33-6ad7dccf49a4 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:11.889671 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 36/69] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:11.895444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6e0c1-6616-4287-af33-6ad7dccf49a4 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:11.900016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93faf623-1a4b-4ce9-8a9f-1151c8597100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-93faf623-1a4b-4ce9-8a9f-1151c8597100 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:11.905224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:11.906099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:11.907177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:11.907177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:11.907177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:11.907177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:11.907177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:11.907177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:11.907177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:11.907177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:11.907177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:11.907177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:11.907177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:11.907177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:11.907177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 34/70] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:52:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:52:11.907838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1974018-ebd8-4c45-9a1c-953c9bcf18b6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:11.907838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1974018-ebd8-4c45-9a1c-953c9bcf18b6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:11.908098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1974018-ebd8-4c45-9a1c-953c9bcf18b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:11.931893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a1974018-ebd8-4c45-9a1c-953c9bcf18b6 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:11.933408 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a1974018-ebd8-4c45-9a1c-953c9bcf18b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:11.982188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:12.006925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 35/71] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:12.008402 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa063f-c60c-421f-9902-a9c757a2bb57 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:12.008835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa063f-c60c-421f-9902-a9c757a2bb57 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:12.009838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa063f-c60c-421f-9902-a9c757a2bb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:12.028626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f7aa063f-c60c-421f-9902-a9c757a2bb57 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:12.030055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f7aa063f-c60c-421f-9902-a9c757a2bb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:12.051809 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6e0c1-6616-4287-af33-6ad7dccf49a4 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:12.130475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 36/72] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:12.131701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa063f-c60c-421f-9902-a9c757a2bb57 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:12.144874 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9f2b1-e711-4493-be3d-c1e51a470f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:12.145814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9f2b1-e711-4493-be3d-c1e51a470f2e None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:12.146312 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9f2b1-e711-4493-be3d-c1e51a470f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:12.146778 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9f2b1-e711-4493-be3d-c1e51a470f2e None admin] PATH_INFO: `/v3/users/b7b49d2a1ef74401b3242303d28a3f46` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:12.163497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9f2b1-e711-4493-be3d-c1e51a470f2e None admin] RBAC: Authorizing `identity:get_user(user_id=b7b49d2a1ef74401b3242303d28a3f46)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:12.166187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9f2b1-e711-4493-be3d-c1e51a470f2e None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:12.168145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 37/73] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:11 2022] GET /identity/v3/users/b7b49d2a1ef74401b3242303d28a3f46 => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:12.173936 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9f2b1-e711-4493-be3d-c1e51a470f2e None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:12.208737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:12.288265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa063f-c60c-421f-9902-a9c757a2bb57 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:12.298914 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:12.303056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9f2b1-e711-4493-be3d-c1e51a470f2e None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:12.326305 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b1814-5e0f-43f8-8d27-ac679e764d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:12.326927 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b1814-5e0f-43f8-8d27-ac679e764d6f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:12.327208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b1814-5e0f-43f8-8d27-ac679e764d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:12.327485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b1814-5e0f-43f8-8d27-ac679e764d6f None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:12.328185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b1814-5e0f-43f8-8d27-ac679e764d6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:12.328492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b1814-5e0f-43f8-8d27-ac679e764d6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:12.329883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b1814-5e0f-43f8-8d27-ac679e764d6f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:12.342955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 37/74] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:12.346009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fb984-de4b-4bde-b8c1-30ec32829dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:12.346648 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fb984-de4b-4bde-b8c1-30ec32829dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:12.346931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fb984-de4b-4bde-b8c1-30ec32829dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:12.347191 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fb984-de4b-4bde-b8c1-30ec32829dc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:12.348701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21884569-bccf-4e5e-b641-ee3ff710d848 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:12.348992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21884569-bccf-4e5e-b641-ee3ff710d848 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:12.349268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21884569-bccf-4e5e-b641-ee3ff710d848 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:12.354246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 38/75] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:12.354246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21884569-bccf-4e5e-b641-ee3ff710d848 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:12.425086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fb984-de4b-4bde-b8c1-30ec32829dc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:12.426962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fb984-de4b-4bde-b8c1-30ec32829dc6 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:12.458135 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:12.468572 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21884569-bccf-4e5e-b641-ee3ff710d848 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:12.512639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07edf66-d137-4ac7-ab48-985f15e38ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:12.513280 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07edf66-d137-4ac7-ab48-985f15e38ffe None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:12.513556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07edf66-d137-4ac7-ab48-985f15e38ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:12.513833 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07edf66-d137-4ac7-ab48-985f15e38ffe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:12.528137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07edf66-d137-4ac7-ab48-985f15e38ffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-f07edf66-d137-4ac7-ab48-985f15e38ffe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:12.533161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:12.534778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:12.535959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:12.535959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:12.535959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:12.535959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:12.535959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:12.535959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:12.535959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:12.535959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:12.535959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:12.535959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:12.535959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:12.535959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:12.535959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 39/76] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:52:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:12.535959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07edf66-d137-4ac7-ab48-985f15e38ffe None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:12.545903 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 38/77] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:52:12.547219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533fa5a-eea6-40ae-9c5e-c5508eb85c27 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:12.547505 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533fa5a-eea6-40ae-9c5e-c5508eb85c27 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:12.547779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533fa5a-eea6-40ae-9c5e-c5508eb85c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:12.565411 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-2533fa5a-eea6-40ae-9c5e-c5508eb85c27 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:12.566670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2533fa5a-eea6-40ae-9c5e-c5508eb85c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:12.567618 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:12.633794 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 39/78] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:12.634741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533fa5a-eea6-40ae-9c5e-c5508eb85c27 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:12.664164 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07edf66-d137-4ac7-ab48-985f15e38ffe None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:12.699344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:12.704512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c9329-7965-44d2-b310-e3624465eb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:12.705458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c9329-7965-44d2-b310-e3624465eb47 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:12.705907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c9329-7965-44d2-b310-e3624465eb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:12.706448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c9329-7965-44d2-b310-e3624465eb47 None admin] PATH_INFO: `/v3/users/5c92b48d12534a97833b0661f5d186ee` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:12.708092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c9329-7965-44d2-b310-e3624465eb47 None admin] RBAC: Authorizing `identity:get_user(user_id=5c92b48d12534a97833b0661f5d186ee)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:12.709701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c9329-7965-44d2-b310-e3624465eb47 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:12.713155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 40/79] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:12 2022] GET /identity/v3/users/5c92b48d12534a97833b0661f5d186ee => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:12.714094 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c9329-7965-44d2-b310-e3624465eb47 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:12.742983 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533fa5a-eea6-40ae-9c5e-c5508eb85c27 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:12.781238 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c01d0d-65e7-4211-81e4-3e01b2da1840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:12.781871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c01d0d-65e7-4211-81e4-3e01b2da1840 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:12.782184 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c01d0d-65e7-4211-81e4-3e01b2da1840 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:12.782444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c01d0d-65e7-4211-81e4-3e01b2da1840 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:12.793678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c01d0d-65e7-4211-81e4-3e01b2da1840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-04c01d0d-65e7-4211-81e4-3e01b2da1840 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:12.798816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:12.799755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:12.800516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:12.800516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:12.800516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:12.800516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:12.800516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:12.800516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:12.800516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:12.800516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:12.800516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:12.800516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:12.800516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:12.800516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:12.800516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 40/80] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:12.801154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c01d0d-65e7-4211-81e4-3e01b2da1840 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:12.836691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:12.842179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c9329-7965-44d2-b310-e3624465eb47 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:12.878284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf46a3f-724b-4a35-8174-696af64889df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:12.878901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf46a3f-724b-4a35-8174-696af64889df None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:12.879181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf46a3f-724b-4a35-8174-696af64889df None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:12.879452 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf46a3f-724b-4a35-8174-696af64889df None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:12.880135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf46a3f-724b-4a35-8174-696af64889df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:12.880441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf46a3f-724b-4a35-8174-696af64889df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:12.881834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf46a3f-724b-4a35-8174-696af64889df None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:12.890296 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 41/81] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:52:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:12.891137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf46a3f-724b-4a35-8174-696af64889df None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:12.911049 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c01d0d-65e7-4211-81e4-3e01b2da1840 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:12.932339 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:12.947976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a56b6-2789-47ed-b742-8142f8d26daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:12.948615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a56b6-2789-47ed-b742-8142f8d26daf None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:12.948895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a56b6-2789-47ed-b742-8142f8d26daf None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:12.949216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a56b6-2789-47ed-b742-8142f8d26daf None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:12.950516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a56b6-2789-47ed-b742-8142f8d26daf None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:12.951927 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a56b6-2789-47ed-b742-8142f8d26daf None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:12.953581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 41/82] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:12 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:12.958598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a56b6-2789-47ed-b742-8142f8d26daf None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:13.000653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf46a3f-724b-4a35-8174-696af64889df None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:13.033011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c357cf-a9d5-4bd5-9328-9b49d997e226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:13.033620 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c357cf-a9d5-4bd5-9328-9b49d997e226 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:13.033895 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c357cf-a9d5-4bd5-9328-9b49d997e226 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:13.034193 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c357cf-a9d5-4bd5-9328-9b49d997e226 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:13.054097 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a56b6-2789-47ed-b742-8142f8d26daf None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:13.061981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:13.088948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b4aed-5293-45cb-966e-03218c20c20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:13.089647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b4aed-5293-45cb-966e-03218c20c20c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:13.089829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b4aed-5293-45cb-966e-03218c20c20c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:13.090099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b4aed-5293-45cb-966e-03218c20c20c None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:13.090791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b4aed-5293-45cb-966e-03218c20c20c None admin] RBAC: Adding query filter params (name=member) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:13.091095 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b4aed-5293-45cb-966e-03218c20c20c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:13.092468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b4aed-5293-45cb-966e-03218c20c20c None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:13.101723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 42/83] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:13.102030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c357cf-a9d5-4bd5-9328-9b49d997e226 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:13.102608 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b4aed-5293-45cb-966e-03218c20c20c None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:13.104039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c357cf-a9d5-4bd5-9328-9b49d997e226 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:13.155574 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:13.207282 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b4aed-5293-45cb-966e-03218c20c20c None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:13.210917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 42/84] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:13.211800 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c357cf-a9d5-4bd5-9328-9b49d997e226 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:13.245851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db909021-1912-45f7-b0b9-71200a8e8891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:13.246520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db909021-1912-45f7-b0b9-71200a8e8891 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:13.246806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db909021-1912-45f7-b0b9-71200a8e8891 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:13.247074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db909021-1912-45f7-b0b9-71200a8e8891 None admin] PATH_INFO: `/v3/users/b7b49d2a1ef74401b3242303d28a3f46` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:13.248216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db909021-1912-45f7-b0b9-71200a8e8891 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b49d2a1ef74401b3242303d28a3f46)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:13.249677 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db909021-1912-45f7-b0b9-71200a8e8891 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:13.251318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 43/85] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:13 2022] GET /identity/v3/users/b7b49d2a1ef74401b3242303d28a3f46 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:13.252079 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db909021-1912-45f7-b0b9-71200a8e8891 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:13.275507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:13.315836 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c357cf-a9d5-4bd5-9328-9b49d997e226 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:13.351144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549734e-d399-4bfe-a6a5-a8af7080370e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:13.351433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db909021-1912-45f7-b0b9-71200a8e8891 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:13.351778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549734e-d399-4bfe-a6a5-a8af7080370e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:13.352064 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549734e-d399-4bfe-a6a5-a8af7080370e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:13.352337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549734e-d399-4bfe-a6a5-a8af7080370e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:13.354008 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549734e-d399-4bfe-a6a5-a8af7080370e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:13.355468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549734e-d399-4bfe-a6a5-a8af7080370e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:13.357039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 43/86] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:52:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:13.357815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549734e-d399-4bfe-a6a5-a8af7080370e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:13.369018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:13.389288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46dbcc6-dd95-45bd-a1a9-c757c5b2ed29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:13.389897 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46dbcc6-dd95-45bd-a1a9-c757c5b2ed29 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:13.390206 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46dbcc6-dd95-45bd-a1a9-c757c5b2ed29 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:13.390482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46dbcc6-dd95-45bd-a1a9-c757c5b2ed29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:13.391215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46dbcc6-dd95-45bd-a1a9-c757c5b2ed29 None admin] RBAC: Adding query filter params (role.id=e385f7d11fac4932a910ec0ef95e2a43, scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa, user.id=b7b49d2a1ef74401b3242303d28a3f46) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:13.391521 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46dbcc6-dd95-45bd-a1a9-c757c5b2ed29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:13.392985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46dbcc6-dd95-45bd-a1a9-c757c5b2ed29 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:13.398367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 44/87] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:13 2022] GET /identity/v3/role_assignments?user.id=b7b49d2a1ef74401b3242303d28a3f46&scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa&role.id=e385f7d11fac4932a910ec0ef95e2a43 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:13.399153 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46dbcc6-dd95-45bd-a1a9-c757c5b2ed29 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:13.485145 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549734e-d399-4bfe-a6a5-a8af7080370e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:13.511544 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46dbcc6-dd95-45bd-a1a9-c757c5b2ed29 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:13.521144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:13.531477 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb9c53-0c8c-4d35-b564-007df8ea1378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:13.532139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb9c53-0c8c-4d35-b564-007df8ea1378 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:13.532425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb9c53-0c8c-4d35-b564-007df8ea1378 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:13.532701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb9c53-0c8c-4d35-b564-007df8ea1378 None admin] PATH_INFO: `/v3/users/b7b49d2a1ef74401b3242303d28a3f46` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:13.533906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb9c53-0c8c-4d35-b564-007df8ea1378 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b49d2a1ef74401b3242303d28a3f46)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:13.535453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb9c53-0c8c-4d35-b564-007df8ea1378 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:13.537097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 44/88] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:13 2022] GET /identity/v3/users/b7b49d2a1ef74401b3242303d28a3f46 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:13.537902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb9c53-0c8c-4d35-b564-007df8ea1378 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:13.560148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9354d4-d6ae-4ed7-a50e-384c5c5c150d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:13.560979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9354d4-d6ae-4ed7-a50e-384c5c5c150d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:13.561263 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9354d4-d6ae-4ed7-a50e-384c5c5c150d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:13.561595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9354d4-d6ae-4ed7-a50e-384c5c5c150d None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:13.562852 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9354d4-d6ae-4ed7-a50e-384c5c5c150d None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:13.564578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9354d4-d6ae-4ed7-a50e-384c5c5c150d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:13.571323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 45/89] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:13 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:13.571323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9354d4-d6ae-4ed7-a50e-384c5c5c150d None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:13.627182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:13.669713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb9c53-0c8c-4d35-b564-007df8ea1378 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:13.679483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9354d4-d6ae-4ed7-a50e-384c5c5c150d None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:13.704886 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88758da7-f4b9-48bb-86ba-17bf93800bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:13.705501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88758da7-f4b9-48bb-86ba-17bf93800bdc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:13.705776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88758da7-f4b9-48bb-86ba-17bf93800bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:13.706047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88758da7-f4b9-48bb-86ba-17bf93800bdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:13.715692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3437ab-6b87-4a25-a0a4-b6cc4d8bbbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:13.716302 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3437ab-6b87-4a25-a0a4-b6cc4d8bbbbb None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:13.716582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3437ab-6b87-4a25-a0a4-b6cc4d8bbbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:13.716894 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3437ab-6b87-4a25-a0a4-b6cc4d8bbbbb None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:13.717514 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3437ab-6b87-4a25-a0a4-b6cc4d8bbbbb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:13.719004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3437ab-6b87-4a25-a0a4-b6cc4d8bbbbb None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:13.741197 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 46/90] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:52:13 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:52:13.741968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3437ab-6b87-4a25-a0a4-b6cc4d8bbbbb None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:13.788175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:13.797742 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88758da7-f4b9-48bb-86ba-17bf93800bdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:13.799298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88758da7-f4b9-48bb-86ba-17bf93800bdc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:13.911976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3437ab-6b87-4a25-a0a4-b6cc4d8bbbbb None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:13.923459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:13.937090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 45/91] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:13.939613 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88758da7-f4b9-48bb-86ba-17bf93800bdc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:13.958734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fbc91-019f-494c-8d7d-1d1993439752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:13.958927 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fbc91-019f-494c-8d7d-1d1993439752 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:13.959186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fbc91-019f-494c-8d7d-1d1993439752 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:13.959453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fbc91-019f-494c-8d7d-1d1993439752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:14.041234 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fbc91-019f-494c-8d7d-1d1993439752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:14.042718 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fbc91-019f-494c-8d7d-1d1993439752 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:14.058337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:14.064837 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88758da7-f4b9-48bb-86ba-17bf93800bdc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:14.119642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98914e5-0c5c-4c05-9717-fa90c847e6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:14.121542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98914e5-0c5c-4c05-9717-fa90c847e6cf None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:14.121542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98914e5-0c5c-4c05-9717-fa90c847e6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:14.121542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98914e5-0c5c-4c05-9717-fa90c847e6cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:14.122526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98914e5-0c5c-4c05-9717-fa90c847e6cf None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa, user.id=5c92b48d12534a97833b0661f5d186ee) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:14.123110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98914e5-0c5c-4c05-9717-fa90c847e6cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:14.126444 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98914e5-0c5c-4c05-9717-fa90c847e6cf None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:14.134172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 46/92] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:13 2022] GET /identity/v3/role_assignments?user.id=5c92b48d12534a97833b0661f5d186ee&scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:14.136324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98914e5-0c5c-4c05-9717-fa90c847e6cf None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:14.164625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 47/93] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:14.171510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fbc91-019f-494c-8d7d-1d1993439752 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:14.180822 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:14.268955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98914e5-0c5c-4c05-9717-fa90c847e6cf None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:14.296428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fbc91-019f-494c-8d7d-1d1993439752 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:14.307356 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6df17b-cb63-4e7f-b4cb-70c160c88ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:14.308011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6df17b-cb63-4e7f-b4cb-70c160c88ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:14.308289 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6df17b-cb63-4e7f-b4cb-70c160c88ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:14.308561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6df17b-cb63-4e7f-b4cb-70c160c88ae0 None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:14.309861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6df17b-cb63-4e7f-b4cb-70c160c88ae0 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:14.312293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6df17b-cb63-4e7f-b4cb-70c160c88ae0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:14.314331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 47/94] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:14 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:14.320479 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6df17b-cb63-4e7f-b4cb-70c160c88ae0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:14.341973 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2837e01-b28b-4ffc-816f-4273fc166af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:14.342682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2837e01-b28b-4ffc-816f-4273fc166af8 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:14.342953 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2837e01-b28b-4ffc-816f-4273fc166af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:14.343223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2837e01-b28b-4ffc-816f-4273fc166af8 None admin] PATH_INFO: `/v3/projects/3e9a223f56de444799c9cdfa4383f99c` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:14.344471 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2837e01-b28b-4ffc-816f-4273fc166af8 None admin] RBAC: Authorizing `identity:get_project(project_id=3e9a223f56de444799c9cdfa4383f99c)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:14.346329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2837e01-b28b-4ffc-816f-4273fc166af8 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:14.348052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 48/95] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:14 2022] GET /identity/v3/projects/3e9a223f56de444799c9cdfa4383f99c => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:14.354860 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2837e01-b28b-4ffc-816f-4273fc166af8 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:14.357263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:14.453263 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:14.485974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2837e01-b28b-4ffc-816f-4273fc166af8 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:14.519883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6df17b-cb63-4e7f-b4cb-70c160c88ae0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:14.562862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a5a19-eaf8-4607-9ba1-ac1c935e0c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:14.563409 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a5a19-eaf8-4607-9ba1-ac1c935e0c1f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:14.563978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a5a19-eaf8-4607-9ba1-ac1c935e0c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:14.564254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a5a19-eaf8-4607-9ba1-ac1c935e0c1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:14.565031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a5a19-eaf8-4607-9ba1-ac1c935e0c1f None admin] RBAC: Adding query filter params (role.id=d43148a29472443ba7dfe2424ec60baf, scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa, user.id=b7b49d2a1ef74401b3242303d28a3f46) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:14.565341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a5a19-eaf8-4607-9ba1-ac1c935e0c1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:14.566867 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a5a19-eaf8-4607-9ba1-ac1c935e0c1f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:14.574570 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 48/96] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:14 2022] GET /identity/v3/role_assignments?user.id=b7b49d2a1ef74401b3242303d28a3f46&scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa&role.id=d43148a29472443ba7dfe2424ec60baf => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:14.599681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16320a-9fd1-4245-a6e6-7260f1e59acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:14.600324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16320a-9fd1-4245-a6e6-7260f1e59acd None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:14.600610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16320a-9fd1-4245-a6e6-7260f1e59acd None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:14.600891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16320a-9fd1-4245-a6e6-7260f1e59acd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:14.601658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16320a-9fd1-4245-a6e6-7260f1e59acd None admin] RBAC: Adding query filter params (role.id=e385f7d11fac4932a910ec0ef95e2a43, scope.project.id=3e9a223f56de444799c9cdfa4383f99c, user.id=b7b49d2a1ef74401b3242303d28a3f46) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:14.601973 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16320a-9fd1-4245-a6e6-7260f1e59acd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:14.603625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16320a-9fd1-4245-a6e6-7260f1e59acd None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:14.614854 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 49/97] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:14 2022] GET /identity/v3/role_assignments?user.id=b7b49d2a1ef74401b3242303d28a3f46&scope.project.id=3e9a223f56de444799c9cdfa4383f99c&role.id=e385f7d11fac4932a910ec0ef95e2a43 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:16.343790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc215b-8b07-45ed-a118-31cc7c4d41eb None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:16.344584 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc215b-8b07-45ed-a118-31cc7c4d41eb None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:16.344584 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc215b-8b07-45ed-a118-31cc7c4d41eb None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:16.345896 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 49/98] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:52:16.351080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a29dcb-0a44-47e7-9cc6-90b37cfaf9ce None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:16.351767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a29dcb-0a44-47e7-9cc6-90b37cfaf9ce None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:16.351767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a29dcb-0a44-47e7-9cc6-90b37cfaf9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:16.376165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f5a29dcb-0a44-47e7-9cc6-90b37cfaf9ce None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:16.377356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f5a29dcb-0a44-47e7-9cc6-90b37cfaf9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:16.477178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 50/99] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:17.065334 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc215b-8b07-45ed-a118-31cc7c4d41eb None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:17.238111 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:17.260244 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc215b-8b07-45ed-a118-31cc7c4d41eb None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:17.327068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd992c-0a6a-4e88-90dc-cca428834454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:17.327974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd992c-0a6a-4e88-90dc-cca428834454 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:17.328418 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd992c-0a6a-4e88-90dc-cca428834454 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:17.328859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd992c-0a6a-4e88-90dc-cca428834454 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:17.347782 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd992c-0a6a-4e88-90dc-cca428834454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-8bdd992c-0a6a-4e88-90dc-cca428834454 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:17.357480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:17.358688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:17.359525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:17.359525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:17.359525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:17.359525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:17.359525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:17.359525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:17.359525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:17.359525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:17.359525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:17.359525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:17.359525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:17.359525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:17.360097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 50/100] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:17.367210 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a29dcb-0a44-47e7-9cc6-90b37cfaf9ce None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:17.477024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:17.526735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a29dcb-0a44-47e7-9cc6-90b37cfaf9ce None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:17.543248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e8378-336f-47c0-9e8b-3013a9beb1ec None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:17.543745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e8378-336f-47c0-9e8b-3013a9beb1ec None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:17.544277 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e8378-336f-47c0-9e8b-3013a9beb1ec None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:17.547442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 51/101] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:17.554592 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864301c-5422-419a-8d57-9fdbc400c41c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:17.555111 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864301c-5422-419a-8d57-9fdbc400c41c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:17.555612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864301c-5422-419a-8d57-9fdbc400c41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:17.616203 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b864301c-5422-419a-8d57-9fdbc400c41c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:17.618149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b864301c-5422-419a-8d57-9fdbc400c41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:17.640481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2664cc4-ee14-4ac7-bc91-db08ae543f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:17.641473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2664cc4-ee14-4ac7-bc91-db08ae543f87 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:17.641938 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2664cc4-ee14-4ac7-bc91-db08ae543f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:17.642431 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2664cc4-ee14-4ac7-bc91-db08ae543f87 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:17.643577 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2664cc4-ee14-4ac7-bc91-db08ae543f87 None admin] RBAC: Adding query filter params (name=member) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:17.644088 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2664cc4-ee14-4ac7-bc91-db08ae543f87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:17.646576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2664cc4-ee14-4ac7-bc91-db08ae543f87 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:17.677674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 51/102] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:17.679868 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a3388-a79d-47b4-b4fa-c79f0ed0b1af None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:17.680333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a3388-a79d-47b4-b4fa-c79f0ed0b1af None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:17.680806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a3388-a79d-47b4-b4fa-c79f0ed0b1af None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:17.684277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 52/103] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:17.686614 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c30f7-7a53-43be-b540-8275261c044e None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:17.687678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c30f7-7a53-43be-b540-8275261c044e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:17.688561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c30f7-7a53-43be-b540-8275261c044e None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:17.691922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 53/104] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:17.693154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c30f7-7a53-43be-b540-8275261c044e None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:17.774340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 52/105] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:17.777898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732cacd-f6c8-481d-bdd4-1d1199ba1eb2 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:17.778423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732cacd-f6c8-481d-bdd4-1d1199ba1eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:17.778970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732cacd-f6c8-481d-bdd4-1d1199ba1eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:17.834994 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a732cacd-f6c8-481d-bdd4-1d1199ba1eb2 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:17.837000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a732cacd-f6c8-481d-bdd4-1d1199ba1eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:17.962962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c30f7-7a53-43be-b540-8275261c044e None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:17.979602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 53/106] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:17.981284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63faaf-19c5-483c-98b5-96408dcb4f50 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:17.981767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63faaf-19c5-483c-98b5-96408dcb4f50 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:17.982273 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63faaf-19c5-483c-98b5-96408dcb4f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:18.003706 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-cd63faaf-19c5-483c-98b5-96408dcb4f50 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:18.005204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cd63faaf-19c5-483c-98b5-96408dcb4f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:18.029998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:18.030209 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:18.044995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0dee58-0526-499f-b25e-727419608f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:18.046680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0dee58-0526-499f-b25e-727419608f99 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:18.047301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0dee58-0526-499f-b25e-727419608f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:18.048212 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0dee58-0526-499f-b25e-727419608f99 None admin] PATH_INFO: `/v3/users/b7b49d2a1ef74401b3242303d28a3f46` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:18.050873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0dee58-0526-499f-b25e-727419608f99 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b49d2a1ef74401b3242303d28a3f46)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:18.056776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0dee58-0526-499f-b25e-727419608f99 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:18.058865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 54/107] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:17 2022] GET /identity/v3/users/b7b49d2a1ef74401b3242303d28a3f46 => generated 275 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:18.061507 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8be5d-6f97-491e-b5fc-4147eac9a7e8 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:18.062830 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8be5d-6f97-491e-b5fc-4147eac9a7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:18.063452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8be5d-6f97-491e-b5fc-4147eac9a7e8 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:18.066855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 55/108] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:18.068982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8be5d-6f97-491e-b5fc-4147eac9a7e8 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:18.206437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 54/109] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:18.208630 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4eb6a2-9398-47fa-b94a-d9c032a6021a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:18.209123 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4eb6a2-9398-47fa-b94a-d9c032a6021a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:18.209609 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4eb6a2-9398-47fa-b94a-d9c032a6021a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:18.266743 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-bb4eb6a2-9398-47fa-b94a-d9c032a6021a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:18.268685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bb4eb6a2-9398-47fa-b94a-d9c032a6021a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:18.276415 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8be5d-6f97-491e-b5fc-4147eac9a7e8 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:18.320958 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dea035-4716-4280-8ea6-ca824761ab3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:18.322579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dea035-4716-4280-8ea6-ca824761ab3b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:18.323081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dea035-4716-4280-8ea6-ca824761ab3b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:18.323548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dea035-4716-4280-8ea6-ca824761ab3b None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:18.325035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dea035-4716-4280-8ea6-ca824761ab3b None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:18.326890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dea035-4716-4280-8ea6-ca824761ab3b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:18.328736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 56/110] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:18 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:18.337931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dea035-4716-4280-8ea6-ca824761ab3b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:18.410194 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:18.453551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 55/111] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:18.455730 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dd225-d7ae-4b69-8fd7-ecd48ab9d90a None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:18.456237 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dd225-d7ae-4b69-8fd7-ecd48ab9d90a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:18.456747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dd225-d7ae-4b69-8fd7-ecd48ab9d90a None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:18.459143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 56/112] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:18.459143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dd225-d7ae-4b69-8fd7-ecd48ab9d90a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:18.465347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dea035-4716-4280-8ea6-ca824761ab3b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:18.524563 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19223f05-b558-4e5a-924a-d48f798faab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:18.525633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19223f05-b558-4e5a-924a-d48f798faab9 None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:18.526138 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19223f05-b558-4e5a-924a-d48f798faab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:18.526647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19223f05-b558-4e5a-924a-d48f798faab9 None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa/users/b7b49d2a1ef74401b3242303d28a3f46/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:18.530040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19223f05-b558-4e5a-924a-d48f798faab9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f1f1e3af21d472cb21dc183a536cdfa, user_id=b7b49d2a1ef74401b3242303d28a3f46, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:18.532945 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19223f05-b558-4e5a-924a-d48f798faab9 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:18.562054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 57/113] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:52:18 2022] PUT /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa/users/b7b49d2a1ef74401b3242303d28a3f46/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:52:18.563327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19223f05-b558-4e5a-924a-d48f798faab9 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:18.564559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:18.607574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dd225-d7ae-4b69-8fd7-ecd48ab9d90a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:18.654446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c405a-b5e1-4464-8a54-1adc03885b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:18.655483 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c405a-b5e1-4464-8a54-1adc03885b8e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:18.655981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c405a-b5e1-4464-8a54-1adc03885b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:18.656470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c405a-b5e1-4464-8a54-1adc03885b8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:18.658621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c405a-b5e1-4464-8a54-1adc03885b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:18.660566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c405a-b5e1-4464-8a54-1adc03885b8e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:18.662615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 57/114] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:52:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:18.664369 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb30521-aa0d-4084-ac98-6cf08c903a03 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:18.665068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb30521-aa0d-4084-ac98-6cf08c903a03 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:18.666157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb30521-aa0d-4084-ac98-6cf08c903a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:18.699573 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1cb30521-aa0d-4084-ac98-6cf08c903a03 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:18.703245 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1cb30521-aa0d-4084-ac98-6cf08c903a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:18.747632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19223f05-b558-4e5a-924a-d48f798faab9 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:18.780269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:18.793333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ea7f0-84e9-4656-b2c0-cb325a899f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:18.793976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ea7f0-84e9-4656-b2c0-cb325a899f6c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:18.795199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ea7f0-84e9-4656-b2c0-cb325a899f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:18.795199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ea7f0-84e9-4656-b2c0-cb325a899f6c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:18.814505 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ea7f0-84e9-4656-b2c0-cb325a899f6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:18.814714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 58/115] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:18.814714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb30521-aa0d-4084-ac98-6cf08c903a03 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-ff4ea7f0-84e9-4656-b2c0-cb325a899f6c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:18.820256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:18.821322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:18.822137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:18.822137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:18.822137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:18.822137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:18.822137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:18.822137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:18.822137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:18.822137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:18.822137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:18.822137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:18.822137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:18.822137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:18.822137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 58/116] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:18.822855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ea7f0-84e9-4656-b2c0-cb325a899f6c None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:18.922541 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:18.946529 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb30521-aa0d-4084-ac98-6cf08c903a03 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:18.968982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ea7f0-84e9-4656-b2c0-cb325a899f6c None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:18.987653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cd586-242f-4069-b96e-77a17db70aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:18.988965 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cd586-242f-4069-b96e-77a17db70aae None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:18.989650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cd586-242f-4069-b96e-77a17db70aae None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:18.990067 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cd586-242f-4069-b96e-77a17db70aae None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:18.991017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cd586-242f-4069-b96e-77a17db70aae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:18.992721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cd586-242f-4069-b96e-77a17db70aae None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:19.011283 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e2035-6545-4bff-b260-ff8354f0527e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:19.012331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e2035-6545-4bff-b260-ff8354f0527e None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:19.012628 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e2035-6545-4bff-b260-ff8354f0527e None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:19.012957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e2035-6545-4bff-b260-ff8354f0527e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:19.025914 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e2035-6545-4bff-b260-ff8354f0527e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:19.026570 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 59/117] 10.222.0.62 () {60 vars in 1141 bytes} [Fri Sep 9 11:52:18 2022] POST /identity/v3/projects => generated 302 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-597e2035-6545-4bff-b260-ff8354f0527e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:19.033723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:19.034771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:19.035524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:19.035524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:19.035524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:19.035524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:19.035524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:19.035524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:19.035524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:19.035524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:19.035524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:19.035524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:19.035524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:19.035524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:19.035524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 59/118] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:19.035524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e2035-6545-4bff-b260-ff8354f0527e None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:19.038238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cd586-242f-4069-b96e-77a17db70aae None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:19.113485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:19.205140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e2035-6545-4bff-b260-ff8354f0527e None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:19.216604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cd586-242f-4069-b96e-77a17db70aae None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:19.249952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:19.252292 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388ead7-ffcd-457b-a9c6-9f6f42f91ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:19.253476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388ead7-ffcd-457b-a9c6-9f6f42f91ebd None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:19.253963 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388ead7-ffcd-457b-a9c6-9f6f42f91ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:19.254641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388ead7-ffcd-457b-a9c6-9f6f42f91ebd None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:19.255520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388ead7-ffcd-457b-a9c6-9f6f42f91ebd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:19.256019 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388ead7-ffcd-457b-a9c6-9f6f42f91ebd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:19.257621 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388ead7-ffcd-457b-a9c6-9f6f42f91ebd None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:19.268322 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343934e-ec1c-442a-9117-6d1aadf88d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:19.268745 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 60/119] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:19.268965 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343934e-ec1c-442a-9117-6d1aadf88d41 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:19.269235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343934e-ec1c-442a-9117-6d1aadf88d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:19.269504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343934e-ec1c-442a-9117-6d1aadf88d41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:19.270447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388ead7-ffcd-457b-a9c6-9f6f42f91ebd None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:19.282901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343934e-ec1c-442a-9117-6d1aadf88d41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-2343934e-ec1c-442a-9117-6d1aadf88d41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:19.289780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:19.290781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:19.292026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:19.292026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:19.292026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:19.292026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:19.292026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:19.292026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:19.292026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:19.292026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:19.292026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:19.292026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:19.292026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:19.292026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:19.292026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 60/120] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:52:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:19.292026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343934e-ec1c-442a-9117-6d1aadf88d41 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:19.411459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388ead7-ffcd-457b-a9c6-9f6f42f91ebd None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:19.415273 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343934e-ec1c-442a-9117-6d1aadf88d41 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:19.426501 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:19.468054 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7bc59b-6117-4bf5-8044-b0aad4a04cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:19.468826 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bc59b-6117-4bf5-8044-b0aad4a04cbb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:19.469181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bc59b-6117-4bf5-8044-b0aad4a04cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:19.469523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bc59b-6117-4bf5-8044-b0aad4a04cbb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:19.476643 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe698a-851d-4433-80af-583e5982b508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:19.477755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe698a-851d-4433-80af-583e5982b508 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:19.478786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe698a-851d-4433-80af-583e5982b508 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:19.479541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe698a-851d-4433-80af-583e5982b508 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:19.480446 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe698a-851d-4433-80af-583e5982b508 None admin] RBAC: Adding query filter params (name=member) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:19.480936 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe698a-851d-4433-80af-583e5982b508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:19.482391 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7bc59b-6117-4bf5-8044-b0aad4a04cbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:19.483099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe698a-851d-4433-80af-583e5982b508 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-2e7bc59b-6117-4bf5-8044-b0aad4a04cbb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:19.489130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:19.490025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:19.491045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:19.491045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:19.491045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:19.491045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:19.491045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:19.491045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:19.491045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:19.491045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:19.491045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:19.491045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:19.491045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:19.491045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:19.491045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 61/121] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:52:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:19.491868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7bc59b-6117-4bf5-8044-b0aad4a04cbb None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:19.496676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 61/122] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:19.498790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe698a-851d-4433-80af-583e5982b508 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:19.546199 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:19.614652 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7bc59b-6117-4bf5-8044-b0aad4a04cbb None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:19.629287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe698a-851d-4433-80af-583e5982b508 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:19.670179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054d295-ced7-4bb0-a7b4-bbc921e26679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:19.671227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054d295-ced7-4bb0-a7b4-bbc921e26679 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:19.671508 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054d295-ced7-4bb0-a7b4-bbc921e26679 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:19.671781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054d295-ced7-4bb0-a7b4-bbc921e26679 None admin] PATH_INFO: `/v3/users/5d6589676dfc4249ad16e0ef65b56ddb` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:19.677315 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439b7c4-f76d-4e9c-888f-2db8844ba744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:19.678170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439b7c4-f76d-4e9c-888f-2db8844ba744 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:19.678620 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439b7c4-f76d-4e9c-888f-2db8844ba744 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:19.679064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439b7c4-f76d-4e9c-888f-2db8844ba744 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:19.679909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439b7c4-f76d-4e9c-888f-2db8844ba744 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:19.680868 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439b7c4-f76d-4e9c-888f-2db8844ba744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:19.682788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439b7c4-f76d-4e9c-888f-2db8844ba744 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:19.688551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054d295-ced7-4bb0-a7b4-bbc921e26679 None admin] RBAC: Authorizing `identity:get_user(user_id=5d6589676dfc4249ad16e0ef65b56ddb)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:19.690616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054d295-ced7-4bb0-a7b4-bbc921e26679 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:19.692582 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 62/123] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:19 2022] GET /identity/v3/users/5d6589676dfc4249ad16e0ef65b56ddb => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:19.693358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054d295-ced7-4bb0-a7b4-bbc921e26679 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:19.694104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 62/124] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:19.695196 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439b7c4-f76d-4e9c-888f-2db8844ba744 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:19.736451 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:19.822611 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439b7c4-f76d-4e9c-888f-2db8844ba744 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:19.838234 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:19.843338 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054d295-ced7-4bb0-a7b4-bbc921e26679 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:19.863630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8dc88-4b67-470f-bebe-b904a9c511ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:19.864263 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8dc88-4b67-470f-bebe-b904a9c511ff None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:19.864546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8dc88-4b67-470f-bebe-b904a9c511ff None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:19.864806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8dc88-4b67-470f-bebe-b904a9c511ff None admin] PATH_INFO: `/v3/users/b7b49d2a1ef74401b3242303d28a3f46` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:19.865953 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8dc88-4b67-470f-bebe-b904a9c511ff None admin] RBAC: Authorizing `identity:get_user(user_id=b7b49d2a1ef74401b3242303d28a3f46)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:19.867647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8dc88-4b67-470f-bebe-b904a9c511ff None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:19.869403 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 63/125] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:19 2022] GET /identity/v3/users/b7b49d2a1ef74401b3242303d28a3f46 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:19.870110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8dc88-4b67-470f-bebe-b904a9c511ff None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:19.892217 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd59d0-15c8-4332-92ea-7803ed5cc470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:19.893725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd59d0-15c8-4332-92ea-7803ed5cc470 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:19.894463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd59d0-15c8-4332-92ea-7803ed5cc470 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:19.894929 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd59d0-15c8-4332-92ea-7803ed5cc470 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:19.896042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd59d0-15c8-4332-92ea-7803ed5cc470 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:19.896944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd59d0-15c8-4332-92ea-7803ed5cc470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:19.898593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd59d0-15c8-4332-92ea-7803ed5cc470 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:19.911860 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 63/126] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:52:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:19.916645 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd59d0-15c8-4332-92ea-7803ed5cc470 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:19.989049 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:19.992592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8dc88-4b67-470f-bebe-b904a9c511ff None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:20.039974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d2058-d5d3-4f8c-8555-c3e215e67f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:20.040776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d2058-d5d3-4f8c-8555-c3e215e67f57 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:20.041158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d2058-d5d3-4f8c-8555-c3e215e67f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:20.041552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d2058-d5d3-4f8c-8555-c3e215e67f57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:20.055664 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd59d0-15c8-4332-92ea-7803ed5cc470 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:20.109982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a503bd4-6c66-4b3b-a5e3-3052f1bbd98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:20.111059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a503bd4-6c66-4b3b-a5e3-3052f1bbd98d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:20.111446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a503bd4-6c66-4b3b-a5e3-3052f1bbd98d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:20.112205 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a503bd4-6c66-4b3b-a5e3-3052f1bbd98d None admin] PATH_INFO: `/v3/users/5c92b48d12534a97833b0661f5d186ee` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:20.113555 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a503bd4-6c66-4b3b-a5e3-3052f1bbd98d None admin] RBAC: Authorizing `identity:get_user(user_id=5c92b48d12534a97833b0661f5d186ee)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:20.115740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a503bd4-6c66-4b3b-a5e3-3052f1bbd98d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:20.117750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 64/127] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:19 2022] GET /identity/v3/users/5c92b48d12534a97833b0661f5d186ee => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:20.122151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a503bd4-6c66-4b3b-a5e3-3052f1bbd98d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:20.139475 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d2058-d5d3-4f8c-8555-c3e215e67f57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:20.141913 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d2058-d5d3-4f8c-8555-c3e215e67f57 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:20.155704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:20.242289 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a503bd4-6c66-4b3b-a5e3-3052f1bbd98d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:20.256878 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:20.267384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 64/128] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:20.268500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d2058-d5d3-4f8c-8555-c3e215e67f57 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:20.286238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e123d4-bbbf-4de3-8de2-e4eac909d79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:20.287107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e123d4-bbbf-4de3-8de2-e4eac909d79f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:20.287868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e123d4-bbbf-4de3-8de2-e4eac909d79f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:20.288459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e123d4-bbbf-4de3-8de2-e4eac909d79f None admin] PATH_INFO: `/v3/projects/3e9a223f56de444799c9cdfa4383f99c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:20.289862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e123d4-bbbf-4de3-8de2-e4eac909d79f None admin] RBAC: Authorizing `identity:get_project(project_id=3e9a223f56de444799c9cdfa4383f99c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:20.291543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e123d4-bbbf-4de3-8de2-e4eac909d79f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:20.293620 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 65/129] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:20 2022] GET /identity/v3/projects/3e9a223f56de444799c9cdfa4383f99c => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:20.294558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e123d4-bbbf-4de3-8de2-e4eac909d79f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:20.394270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d2058-d5d3-4f8c-8555-c3e215e67f57 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:20.406808 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:20.413463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e123d4-bbbf-4de3-8de2-e4eac909d79f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:20.430504 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9bcd2-c913-4c2d-ac83-33957c018bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:20.431894 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9bcd2-c913-4c2d-ac83-33957c018bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:20.431894 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9bcd2-c913-4c2d-ac83-33957c018bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:20.432164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9bcd2-c913-4c2d-ac83-33957c018bb7 None admin] PATH_INFO: `/v3/users/b7b49d2a1ef74401b3242303d28a3f46` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:20.433731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9bcd2-c913-4c2d-ac83-33957c018bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b49d2a1ef74401b3242303d28a3f46)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:20.434816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9bcd2-c913-4c2d-ac83-33957c018bb7 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:20.436437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 65/130] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:20 2022] GET /identity/v3/users/b7b49d2a1ef74401b3242303d28a3f46 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:20.437213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9bcd2-c913-4c2d-ac83-33957c018bb7 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:20.462767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d4cbe-fbbf-49d2-ba0d-6c9d1b469884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:20.463796 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4cbe-fbbf-49d2-ba0d-6c9d1b469884 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:20.464279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4cbe-fbbf-49d2-ba0d-6c9d1b469884 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:20.464780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4cbe-fbbf-49d2-ba0d-6c9d1b469884 None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:20.466781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d4cbe-fbbf-49d2-ba0d-6c9d1b469884 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:20.469984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d4cbe-fbbf-49d2-ba0d-6c9d1b469884 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:20.472820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 66/131] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:20 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:20.473924 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d4cbe-fbbf-49d2-ba0d-6c9d1b469884 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:20.527243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:20.565392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9bcd2-c913-4c2d-ac83-33957c018bb7 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:20.600382 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d4cbe-fbbf-49d2-ba0d-6c9d1b469884 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:20.611459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8cce3-5a73-49eb-83c8-7fa48e800e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:20.612340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8cce3-5a73-49eb-83c8-7fa48e800e6d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:20.613159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8cce3-5a73-49eb-83c8-7fa48e800e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:20.613582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8cce3-5a73-49eb-83c8-7fa48e800e6d None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:20.615190 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8cce3-5a73-49eb-83c8-7fa48e800e6d None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:20.617704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8cce3-5a73-49eb-83c8-7fa48e800e6d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:20.620486 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 66/132] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:20 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:20.621780 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8cce3-5a73-49eb-83c8-7fa48e800e6d None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:20.644499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f97d0e-1b73-4ee0-94e1-dd878e2e0dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:20.645230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f97d0e-1b73-4ee0-94e1-dd878e2e0dbd None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:20.645647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f97d0e-1b73-4ee0-94e1-dd878e2e0dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:20.646050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f97d0e-1b73-4ee0-94e1-dd878e2e0dbd None admin] PATH_INFO: `/v3/projects/3e9a223f56de444799c9cdfa4383f99c/users/b7b49d2a1ef74401b3242303d28a3f46/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:20.648214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f97d0e-1b73-4ee0-94e1-dd878e2e0dbd None admin] RBAC: Authorizing `identity:create_grant(project_id=3e9a223f56de444799c9cdfa4383f99c, user_id=b7b49d2a1ef74401b3242303d28a3f46, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:20.649810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f97d0e-1b73-4ee0-94e1-dd878e2e0dbd None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:20.669939 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 67/133] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:52:20 2022] PUT /identity/v3/projects/3e9a223f56de444799c9cdfa4383f99c/users/b7b49d2a1ef74401b3242303d28a3f46/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:52:20.671027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f97d0e-1b73-4ee0-94e1-dd878e2e0dbd None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:20.707414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:20.762313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8cce3-5a73-49eb-83c8-7fa48e800e6d None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:20.808234 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bb0d5-9aa6-4c9d-9179-8d4235bfd611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:20.809483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bb0d5-9aa6-4c9d-9179-8d4235bfd611 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:20.809987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bb0d5-9aa6-4c9d-9179-8d4235bfd611 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:20.810460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bb0d5-9aa6-4c9d-9179-8d4235bfd611 None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:20.812066 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bb0d5-9aa6-4c9d-9179-8d4235bfd611 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:20.815129 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bb0d5-9aa6-4c9d-9179-8d4235bfd611 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:20.817484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 67/134] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:20 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:20.818612 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bb0d5-9aa6-4c9d-9179-8d4235bfd611 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:20.820626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f97d0e-1b73-4ee0-94e1-dd878e2e0dbd None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:20.823374 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:20.862527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586e510-0d01-4685-be56-0cb50fea814b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:20.863164 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586e510-0d01-4685-be56-0cb50fea814b None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:20.863453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586e510-0d01-4685-be56-0cb50fea814b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:20.863734 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586e510-0d01-4685-be56-0cb50fea814b None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa/users/5c92b48d12534a97833b0661f5d186ee/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:20.867238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586e510-0d01-4685-be56-0cb50fea814b None admin] RBAC: Authorizing `identity:create_grant(project_id=4f1f1e3af21d472cb21dc183a536cdfa, user_id=5c92b48d12534a97833b0661f5d186ee, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:20.869482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586e510-0d01-4685-be56-0cb50fea814b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:20.886026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 68/135] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:52:20 2022] PUT /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa/users/5c92b48d12534a97833b0661f5d186ee/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:52:20.887072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586e510-0d01-4685-be56-0cb50fea814b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:20.942556 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bb0d5-9aa6-4c9d-9179-8d4235bfd611 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:20.981899 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0f496-5473-4ea8-b486-6a3d2025bcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:20.982944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0f496-5473-4ea8-b486-6a3d2025bcdf None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:20.983679 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0f496-5473-4ea8-b486-6a3d2025bcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:20.984173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0f496-5473-4ea8-b486-6a3d2025bcdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:20.985139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0f496-5473-4ea8-b486-6a3d2025bcdf None admin] RBAC: Adding query filter params (role.id=80400d1e5fdb44b3a92a468f6b62b248, scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa, user.id=5d6589676dfc4249ad16e0ef65b56ddb) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:20.985663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0f496-5473-4ea8-b486-6a3d2025bcdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:20.987410 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0f496-5473-4ea8-b486-6a3d2025bcdf None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:20.993905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 68/136] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:20 2022] GET /identity/v3/role_assignments?user.id=5d6589676dfc4249ad16e0ef65b56ddb&scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa&role.id=80400d1e5fdb44b3a92a468f6b62b248 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:20.996979 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:21.023923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586e510-0d01-4685-be56-0cb50fea814b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:21.061027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a08808-62f3-4db7-827d-9e687cedc2eb None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:21.061761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a08808-62f3-4db7-827d-9e687cedc2eb None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:21.062260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a08808-62f3-4db7-827d-9e687cedc2eb None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:21.064296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 69/137] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:52:21.070784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c97dad-47d1-467b-a9e8-5ddf1b8b2bd5 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:21.071086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c97dad-47d1-467b-a9e8-5ddf1b8b2bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:21.071370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c97dad-47d1-467b-a9e8-5ddf1b8b2bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:21.073843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bac9e2-7711-4710-9e63-ae02ca1d63f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:21.074587 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bac9e2-7711-4710-9e63-ae02ca1d63f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:21.074877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bac9e2-7711-4710-9e63-ae02ca1d63f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:21.075160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bac9e2-7711-4710-9e63-ae02ca1d63f2 None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa/users/b7b49d2a1ef74401b3242303d28a3f46/roles/d43148a29472443ba7dfe2424ec60baf` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:21.077645 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bac9e2-7711-4710-9e63-ae02ca1d63f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f1f1e3af21d472cb21dc183a536cdfa, user_id=b7b49d2a1ef74401b3242303d28a3f46, role_id=d43148a29472443ba7dfe2424ec60baf)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:21.079550 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bac9e2-7711-4710-9e63-ae02ca1d63f2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:21.092169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d6c97dad-47d1-467b-a9e8-5ddf1b8b2bd5 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:21.093848 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d6c97dad-47d1-467b-a9e8-5ddf1b8b2bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:21.122278 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 69/138] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:52:20 2022] PUT /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa/users/b7b49d2a1ef74401b3242303d28a3f46/roles/d43148a29472443ba7dfe2424ec60baf => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:52:21.196158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:21.197998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 70/139] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:21.495199 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bac9e2-7711-4710-9e63-ae02ca1d63f2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:21.615730 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bac9e2-7711-4710-9e63-ae02ca1d63f2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:21.659642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4b8e9-d608-42e9-94be-79ab50db55f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:21.660287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4b8e9-d608-42e9-94be-79ab50db55f4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:21.660566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4b8e9-d608-42e9-94be-79ab50db55f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:21.660870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4b8e9-d608-42e9-94be-79ab50db55f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:21.677357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4b8e9-d608-42e9-94be-79ab50db55f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-b2c4b8e9-d608-42e9-94be-79ab50db55f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:21.683503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:21.684349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:21.685509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:21.685509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:21.685509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:21.685509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:21.685509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:21.685509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:21.685509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:21.685509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:21.685509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:21.685509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:21.685509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:21.685509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:21.685509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 70/140] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:21.692199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c97dad-47d1-467b-a9e8-5ddf1b8b2bd5 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:21.840516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c97dad-47d1-467b-a9e8-5ddf1b8b2bd5 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:21.856789 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2475639-c183-4a07-8206-b644acbaf5ff None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:21.857093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2475639-c183-4a07-8206-b644acbaf5ff None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:21.857344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2475639-c183-4a07-8206-b644acbaf5ff None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:21.860503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 71/141] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:21.874467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcb9a8-f68c-4761-839f-bded8bd7111a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:21.874748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcb9a8-f68c-4761-839f-bded8bd7111a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:21.875090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcb9a8-f68c-4761-839f-bded8bd7111a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:21.888801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5646f-b3af-43a3-a8d9-87d89de2b095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:21.889431 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5646f-b3af-43a3-a8d9-87d89de2b095 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:21.889707 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5646f-b3af-43a3-a8d9-87d89de2b095 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:21.889975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5646f-b3af-43a3-a8d9-87d89de2b095 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:21.890701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5646f-b3af-43a3-a8d9-87d89de2b095 None admin] RBAC: Adding query filter params (name=member) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:21.891001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5646f-b3af-43a3-a8d9-87d89de2b095 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:21.893618 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5646f-b3af-43a3-a8d9-87d89de2b095 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:21.900800 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:21.908994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 71/142] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:21.911368 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-15dcb9a8-f68c-4761-839f-bded8bd7111a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:21.915611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-15dcb9a8-f68c-4761-839f-bded8bd7111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:21.916609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5646f-b3af-43a3-a8d9-87d89de2b095 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:22.032940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:22.063241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 72/143] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:22.105336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5646f-b3af-43a3-a8d9-87d89de2b095 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:22.158988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621bf3f7-8588-4638-8e68-f42227f5954d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:22.159628 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bf3f7-8588-4638-8e68-f42227f5954d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:22.159911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bf3f7-8588-4638-8e68-f42227f5954d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:22.160187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bf3f7-8588-4638-8e68-f42227f5954d None admin] PATH_INFO: `/v3/users/b7b49d2a1ef74401b3242303d28a3f46` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:22.161332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621bf3f7-8588-4638-8e68-f42227f5954d None admin] RBAC: Authorizing `identity:get_user(user_id=b7b49d2a1ef74401b3242303d28a3f46)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:22.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621bf3f7-8588-4638-8e68-f42227f5954d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:22.164408 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 72/144] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:21 2022] GET /identity/v3/users/b7b49d2a1ef74401b3242303d28a3f46 => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:22.170612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcb9a8-f68c-4761-839f-bded8bd7111a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:22.328050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcb9a8-f68c-4761-839f-bded8bd7111a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:22.370955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a4e6e-928a-4e6c-8363-3abbda5e2a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:22.371917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a4e6e-928a-4e6c-8363-3abbda5e2a6c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:22.372415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a4e6e-928a-4e6c-8363-3abbda5e2a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:22.372886 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a4e6e-928a-4e6c-8363-3abbda5e2a6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:22.379196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:22.466363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a4e6e-928a-4e6c-8363-3abbda5e2a6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:22.468297 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a4e6e-928a-4e6c-8363-3abbda5e2a6c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:22.592741 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 73/145] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:22.598572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621bf3f7-8588-4638-8e68-f42227f5954d None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:22.689831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a4e6e-928a-4e6c-8363-3abbda5e2a6c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:22.759633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:22.781910 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621bf3f7-8588-4638-8e68-f42227f5954d None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:22.824768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:22.842372 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b7cec-5c61-4f89-9205-d92b242aa345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:22.843019 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b7cec-5c61-4f89-9205-d92b242aa345 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:22.843303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b7cec-5c61-4f89-9205-d92b242aa345 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:22.843583 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b7cec-5c61-4f89-9205-d92b242aa345 None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:22.844741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b7cec-5c61-4f89-9205-d92b242aa345 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:22.846227 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b7cec-5c61-4f89-9205-d92b242aa345 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:22.847881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 73/146] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:22 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:22.854351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b7cec-5c61-4f89-9205-d92b242aa345 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:22.890514 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a4e6e-928a-4e6c-8363-3abbda5e2a6c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:22.980017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d10d3f-411c-433b-84ae-dfe008b99f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:22.980964 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d10d3f-411c-433b-84ae-dfe008b99f17 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:22.981439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d10d3f-411c-433b-84ae-dfe008b99f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:22.981911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d10d3f-411c-433b-84ae-dfe008b99f17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:22.986775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d10d3f-411c-433b-84ae-dfe008b99f17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:22.989207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d10d3f-411c-433b-84ae-dfe008b99f17 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:22.991393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 74/147] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:52:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:22.996861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d10d3f-411c-433b-84ae-dfe008b99f17 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:23.039669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b7cec-5c61-4f89-9205-d92b242aa345 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:23.102723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49859aa3-94ab-4a64-9f65-4217307d7e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:23.103793 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49859aa3-94ab-4a64-9f65-4217307d7e94 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:23.104071 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49859aa3-94ab-4a64-9f65-4217307d7e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:23.104341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49859aa3-94ab-4a64-9f65-4217307d7e94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:23.105073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49859aa3-94ab-4a64-9f65-4217307d7e94 None admin] RBAC: Adding query filter params (role.id=e385f7d11fac4932a910ec0ef95e2a43, scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa, user.id=b7b49d2a1ef74401b3242303d28a3f46) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:23.105379 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49859aa3-94ab-4a64-9f65-4217307d7e94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:23.107128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49859aa3-94ab-4a64-9f65-4217307d7e94 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:23.127056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 74/148] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:22 2022] GET /identity/v3/role_assignments?user.id=b7b49d2a1ef74401b3242303d28a3f46&scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa&role.id=e385f7d11fac4932a910ec0ef95e2a43 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:23.156029 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d10d3f-411c-433b-84ae-dfe008b99f17 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:23.158668 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:23.198546 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d92ef2-7f8d-4da6-9963-bd81290803e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:23.199236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d92ef2-7f8d-4da6-9963-bd81290803e9 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:23.199517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d92ef2-7f8d-4da6-9963-bd81290803e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:23.199791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d92ef2-7f8d-4da6-9963-bd81290803e9 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:23.200509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d92ef2-7f8d-4da6-9963-bd81290803e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:23.201989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d92ef2-7f8d-4da6-9963-bd81290803e9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:23.244637 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 75/149] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:52:22 2022] POST /identity/v3/users => generated 283 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:52:23.951654 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518d98e-5466-481d-ab56-90d21ed48dea None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:23.952581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518d98e-5466-481d-ab56-90d21ed48dea None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:23.953044 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518d98e-5466-481d-ab56-90d21ed48dea None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:23.954636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 75/150] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:23.961583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7896551-164f-4090-a3ce-2b1e072fe34d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:23.962520 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7896551-164f-4090-a3ce-2b1e072fe34d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:23.962982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7896551-164f-4090-a3ce-2b1e072fe34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:23.982284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f7896551-164f-4090-a3ce-2b1e072fe34d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:23.984024 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f7896551-164f-4090-a3ce-2b1e072fe34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:24.051229 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29512b9-3894-4585-b566-cba5064b1b69 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:24.051860 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29512b9-3894-4585-b566-cba5064b1b69 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:24.052497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29512b9-3894-4585-b566-cba5064b1b69 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:24.054815 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 76/151] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:24.058084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 76/152] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:24.065063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47af52-b3ee-4a51-8a17-39d522d36c7e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:24.065644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47af52-b3ee-4a51-8a17-39d522d36c7e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:24.066148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47af52-b3ee-4a51-8a17-39d522d36c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:24.089925 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-5b47af52-b3ee-4a51-8a17-39d522d36c7e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:24.091477 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-5b47af52-b3ee-4a51-8a17-39d522d36c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:24.170504 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:24.187666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 77/153] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:24.384617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7896551-164f-4090-a3ce-2b1e072fe34d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:24.398954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea035b76-5a23-4e39-9697-e4ec71c3ae8d None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:24.400465 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea035b76-5a23-4e39-9697-e4ec71c3ae8d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:24.400958 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea035b76-5a23-4e39-9697-e4ec71c3ae8d None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:24.403562 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 78/154] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:24.410680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcc859-23b9-4662-ab04-a21f9ab70561 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:24.412037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcc859-23b9-4662-ab04-a21f9ab70561 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:24.412535 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcc859-23b9-4662-ab04-a21f9ab70561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:24.438963 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-3dbcc859-23b9-4662-ab04-a21f9ab70561 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:24.441204 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-3dbcc859-23b9-4662-ab04-a21f9ab70561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:24.510143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:24.528597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7896551-164f-4090-a3ce-2b1e072fe34d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:24.539592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 79/155] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:24.541313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbcc859-23b9-4662-ab04-a21f9ab70561 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:24.589436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b325dc-7736-40c2-b525-32a9ae3daf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:24.590714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b325dc-7736-40c2-b525-32a9ae3daf87 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:24.591133 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b325dc-7736-40c2-b525-32a9ae3daf87 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:24.591544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b325dc-7736-40c2-b525-32a9ae3daf87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:24.605658 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b325dc-7736-40c2-b525-32a9ae3daf87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-32b325dc-7736-40c2-b525-32a9ae3daf87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:24.612458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:24.613552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:24.614569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:24.614569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:24.614569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:24.614569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:24.614569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:24.614569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:24.614569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:24.614569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:24.614569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:24.614569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:24.614569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:24.614569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:24.617020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 77/156] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:24.620384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c15df5-6d19-4218-8b44-b7cb9d63a951 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:24.620826 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c15df5-6d19-4218-8b44-b7cb9d63a951 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:24.621248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c15df5-6d19-4218-8b44-b7cb9d63a951 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:24.622755 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 78/157] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:24.624019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c15df5-6d19-4218-8b44-b7cb9d63a951 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:24.689347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbcc859-23b9-4662-ab04-a21f9ab70561 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:24.713426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:24.729254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ef6a6-3d43-4453-a5c0-f1d37ef67e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:24.730337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ef6a6-3d43-4453-a5c0-f1d37ef67e47 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:24.731044 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ef6a6-3d43-4453-a5c0-f1d37ef67e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:24.731559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ef6a6-3d43-4453-a5c0-f1d37ef67e47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:24.745549 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ef6a6-3d43-4453-a5c0-f1d37ef67e47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-6e3ef6a6-3d43-4453-a5c0-f1d37ef67e47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:24.751323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:24.752447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:24.753288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:24.753288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:24.753288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:24.753288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:24.753288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:24.753288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:24.753288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:24.753288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:24.753288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:24.753288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:24.753288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:24.753288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:24.753288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 80/158] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:52:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:24.753922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df33354-5afb-4613-ac52-d3547c8d9e5b None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:24.754906 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df33354-5afb-4613-ac52-d3547c8d9e5b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:24.755823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df33354-5afb-4613-ac52-d3547c8d9e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:24.756459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c15df5-6d19-4218-8b44-b7cb9d63a951 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:24.776196 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-5df33354-5afb-4613-ac52-d3547c8d9e5b None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:24.777415 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-5df33354-5afb-4613-ac52-d3547c8d9e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:24.798944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31aaf12-044f-49a2-871b-84a4663153ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:24.799933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31aaf12-044f-49a2-871b-84a4663153ae None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:24.800391 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31aaf12-044f-49a2-871b-84a4663153ae None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:24.800839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31aaf12-044f-49a2-871b-84a4663153ae None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:24.801697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31aaf12-044f-49a2-871b-84a4663153ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:24.802204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31aaf12-044f-49a2-871b-84a4663153ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:24.803880 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31aaf12-044f-49a2-871b-84a4663153ae None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:24.815486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 79/159] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:24.816595 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31aaf12-044f-49a2-871b-84a4663153ae None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:24.833341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:24.861979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 81/160] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:24.862874 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df33354-5afb-4613-ac52-d3547c8d9e5b None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:24.948659 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31aaf12-044f-49a2-871b-84a4663153ae None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:24.999481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d2f3c-d892-4e5d-a6e8-f75bb6127ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:25.000146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d2f3c-d892-4e5d-a6e8-f75bb6127ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:25.000460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d2f3c-d892-4e5d-a6e8-f75bb6127ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:25.000731 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d2f3c-d892-4e5d-a6e8-f75bb6127ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:25.001406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d2f3c-d892-4e5d-a6e8-f75bb6127ac3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:25.001703 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d2f3c-d892-4e5d-a6e8-f75bb6127ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:25.003393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d2f3c-d892-4e5d-a6e8-f75bb6127ac3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:25.005000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df33354-5afb-4613-ac52-d3547c8d9e5b None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:25.013140 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 80/161] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:25.013813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d2f3c-d892-4e5d-a6e8-f75bb6127ac3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:25.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:25.065259 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5acc68-74ba-4bc5-9bda-a238afe938cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:25.065930 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5acc68-74ba-4bc5-9bda-a238afe938cc None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:25.066240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5acc68-74ba-4bc5-9bda-a238afe938cc None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:25.066515 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5acc68-74ba-4bc5-9bda-a238afe938cc None admin] PATH_INFO: `/v3/users/b7b49d2a1ef74401b3242303d28a3f46` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:25.067961 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5acc68-74ba-4bc5-9bda-a238afe938cc None admin] RBAC: Authorizing `identity:get_user(user_id=b7b49d2a1ef74401b3242303d28a3f46)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:25.069565 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5acc68-74ba-4bc5-9bda-a238afe938cc None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:25.071212 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 82/162] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:24 2022] GET /identity/v3/users/b7b49d2a1ef74401b3242303d28a3f46 => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:25.072628 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5acc68-74ba-4bc5-9bda-a238afe938cc None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:25.095734 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:25.123428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d2f3c-d892-4e5d-a6e8-f75bb6127ac3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:25.161040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbc5df-d97e-44c4-b039-472f091ff573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:25.161732 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbc5df-d97e-44c4-b039-472f091ff573 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:25.162009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbc5df-d97e-44c4-b039-472f091ff573 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:25.162309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbc5df-d97e-44c4-b039-472f091ff573 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:25.173459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbc5df-d97e-44c4-b039-472f091ff573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-e0cbc5df-d97e-44c4-b039-472f091ff573 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:25.178982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:25.179883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:25.180665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:25.180665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:25.180665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:25.180665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:25.180665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:25.180665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:25.180665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:25.180665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:25.180665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:25.180665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:25.180665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:25.180665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:25.180665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 81/163] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:52:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:25.181832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbc5df-d97e-44c4-b039-472f091ff573 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:25.284363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5acc68-74ba-4bc5-9bda-a238afe938cc None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:25.308359 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbc5df-d97e-44c4-b039-472f091ff573 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:25.321223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8563720-07f2-411a-94f6-57a461cbca54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:25.321965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8563720-07f2-411a-94f6-57a461cbca54 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:25.322270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8563720-07f2-411a-94f6-57a461cbca54 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:25.322543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8563720-07f2-411a-94f6-57a461cbca54 None admin] PATH_INFO: `/v3/users/5c92b48d12534a97833b0661f5d186ee` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:25.324227 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8563720-07f2-411a-94f6-57a461cbca54 None admin] RBAC: Authorizing `identity:get_user(user_id=5c92b48d12534a97833b0661f5d186ee)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:25.326148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8563720-07f2-411a-94f6-57a461cbca54 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:25.328063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 83/164] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:25 2022] GET /identity/v3/users/5c92b48d12534a97833b0661f5d186ee => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:25.328843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8563720-07f2-411a-94f6-57a461cbca54 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:25.347475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3529e-bf3f-416b-89b0-146f80125a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:25.348093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3529e-bf3f-416b-89b0-146f80125a6a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:25.348368 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3529e-bf3f-416b-89b0-146f80125a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:25.348639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3529e-bf3f-416b-89b0-146f80125a6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:25.351734 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:25.406514 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:25.504531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3529e-bf3f-416b-89b0-146f80125a6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:25.506009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3529e-bf3f-416b-89b0-146f80125a6a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:25.515286 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8563720-07f2-411a-94f6-57a461cbca54 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:25.559701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece3d8c-b834-4d88-8406-4e0f99fd51cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:25.560667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece3d8c-b834-4d88-8406-4e0f99fd51cc None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:25.560942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece3d8c-b834-4d88-8406-4e0f99fd51cc None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:25.561209 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece3d8c-b834-4d88-8406-4e0f99fd51cc None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:25.561889 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece3d8c-b834-4d88-8406-4e0f99fd51cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:25.562215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece3d8c-b834-4d88-8406-4e0f99fd51cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:25.563673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece3d8c-b834-4d88-8406-4e0f99fd51cc None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:25.583696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 84/165] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:52:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:25.584584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece3d8c-b834-4d88-8406-4e0f99fd51cc None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:25.741476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece3d8c-b834-4d88-8406-4e0f99fd51cc None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:25.751935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 82/166] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:25.753125 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3529e-bf3f-416b-89b0-146f80125a6a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:25.758475 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:25.803589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017e875-769e-45be-89b7-c731c8bb1f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:25.804601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017e875-769e-45be-89b7-c731c8bb1f69 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:25.805238 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017e875-769e-45be-89b7-c731c8bb1f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:25.806853 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017e875-769e-45be-89b7-c731c8bb1f69 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:25.820264 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:25.830505 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017e875-769e-45be-89b7-c731c8bb1f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-e017e875-769e-45be-89b7-c731c8bb1f69 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:25.836335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:25.837269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:25.838147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:25.838147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:25.838147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:25.838147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:25.838147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:25.838147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:25.838147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:25.838147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:25.838147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:25.838147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:25.838147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:25.838147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:25.843917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 85/167] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:25.845731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017e875-769e-45be-89b7-c731c8bb1f69 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:25.957119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3529e-bf3f-416b-89b0-146f80125a6a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:26.030323 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c56de2-ff2b-4a62-a9ae-6045d048e40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:26.031746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c56de2-ff2b-4a62-a9ae-6045d048e40b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:26.032218 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c56de2-ff2b-4a62-a9ae-6045d048e40b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:26.032678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c56de2-ff2b-4a62-a9ae-6045d048e40b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:26.053536 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017e875-769e-45be-89b7-c731c8bb1f69 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:26.123213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b3eae-bb14-4bcd-b4a1-e3ef79058272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:26.124221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b3eae-bb14-4bcd-b4a1-e3ef79058272 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:26.124690 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b3eae-bb14-4bcd-b4a1-e3ef79058272 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:26.125917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b3eae-bb14-4bcd-b4a1-e3ef79058272 None admin] PATH_INFO: `/v3/users/b7b49d2a1ef74401b3242303d28a3f46` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:26.127389 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b3eae-bb14-4bcd-b4a1-e3ef79058272 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b49d2a1ef74401b3242303d28a3f46)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:26.129206 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b3eae-bb14-4bcd-b4a1-e3ef79058272 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:26.133454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 86/168] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:25 2022] GET /identity/v3/users/b7b49d2a1ef74401b3242303d28a3f46 => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:26.136556 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b3eae-bb14-4bcd-b4a1-e3ef79058272 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:26.168973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c56de2-ff2b-4a62-a9ae-6045d048e40b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:26.170961 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c56de2-ff2b-4a62-a9ae-6045d048e40b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:26.208410 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:26.247263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:26.326791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b3eae-bb14-4bcd-b4a1-e3ef79058272 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:26.376962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 83/169] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:26.377618 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c56de2-ff2b-4a62-a9ae-6045d048e40b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:26.395691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8323016-7de5-43e6-8db3-b5c42cf68dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:26.396332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8323016-7de5-43e6-8db3-b5c42cf68dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:26.396606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8323016-7de5-43e6-8db3-b5c42cf68dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:26.396872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8323016-7de5-43e6-8db3-b5c42cf68dd7 None admin] PATH_INFO: `/v3/projects/3e9a223f56de444799c9cdfa4383f99c` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:26.398726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8323016-7de5-43e6-8db3-b5c42cf68dd7 None admin] RBAC: Authorizing `identity:get_project(project_id=3e9a223f56de444799c9cdfa4383f99c)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:26.400738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8323016-7de5-43e6-8db3-b5c42cf68dd7 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:26.402606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 87/170] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:26 2022] GET /identity/v3/projects/3e9a223f56de444799c9cdfa4383f99c => generated 312 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:26.403225 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8323016-7de5-43e6-8db3-b5c42cf68dd7 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:26.561762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c56de2-ff2b-4a62-a9ae-6045d048e40b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:26.600613 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ecd6f-ae51-49a4-9eec-33101a366464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:26.601484 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ecd6f-ae51-49a4-9eec-33101a366464 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:26.601985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ecd6f-ae51-49a4-9eec-33101a366464 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:26.602421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ecd6f-ae51-49a4-9eec-33101a366464 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:26.604062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ecd6f-ae51-49a4-9eec-33101a366464 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:26.604507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ecd6f-ae51-49a4-9eec-33101a366464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:26.606032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ecd6f-ae51-49a4-9eec-33101a366464 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:26.607854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:26.612958 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8323016-7de5-43e6-8db3-b5c42cf68dd7 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:26.616315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 84/171] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:26.617618 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:26.617742 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ecd6f-ae51-49a4-9eec-33101a366464 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:26.735865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49978e-7e95-49bd-a793-0f13b5c8367d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:26.736507 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49978e-7e95-49bd-a793-0f13b5c8367d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:26.736693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49978e-7e95-49bd-a793-0f13b5c8367d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:26.736967 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49978e-7e95-49bd-a793-0f13b5c8367d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:26.742061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ecd6f-ae51-49a4-9eec-33101a366464 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:26.798854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c7efa-bbab-4132-9feb-68f8741be376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:26.799500 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c7efa-bbab-4132-9feb-68f8741be376 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:26.799880 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c7efa-bbab-4132-9feb-68f8741be376 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:26.800160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c7efa-bbab-4132-9feb-68f8741be376 None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:26.801438 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c7efa-bbab-4132-9feb-68f8741be376 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:26.802927 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c7efa-bbab-4132-9feb-68f8741be376 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:26.804945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 85/172] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:26 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:26.811215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c7efa-bbab-4132-9feb-68f8741be376 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:26.827882 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d49978e-7e95-49bd-a793-0f13b5c8367d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:26.829407 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d49978e-7e95-49bd-a793-0f13b5c8367d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:26.899016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:26.954170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 88/173] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:26.954170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49978e-7e95-49bd-a793-0f13b5c8367d None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:26.957064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:26.960683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c7efa-bbab-4132-9feb-68f8741be376 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:27.021834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c3dae-cbb3-4ed2-8458-7caceb909f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:27.021834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c3dae-cbb3-4ed2-8458-7caceb909f39 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:27.021834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c3dae-cbb3-4ed2-8458-7caceb909f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:27.021834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c3dae-cbb3-4ed2-8458-7caceb909f39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:27.028046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c3dae-cbb3-4ed2-8458-7caceb909f39 None admin] RBAC: Adding query filter params (role.id=e385f7d11fac4932a910ec0ef95e2a43, scope.project.id=3e9a223f56de444799c9cdfa4383f99c, user.id=b7b49d2a1ef74401b3242303d28a3f46) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:27.028046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c3dae-cbb3-4ed2-8458-7caceb909f39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:27.028046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c3dae-cbb3-4ed2-8458-7caceb909f39 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:27.038769 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 86/174] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:26 2022] GET /identity/v3/role_assignments?user.id=b7b49d2a1ef74401b3242303d28a3f46&scope.project.id=3e9a223f56de444799c9cdfa4383f99c&role.id=e385f7d11fac4932a910ec0ef95e2a43 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:27.041713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0e0ef-dc2a-4228-a746-3e5d271642dd None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:27.042704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0e0ef-dc2a-4228-a746-3e5d271642dd None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:27.043366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0e0ef-dc2a-4228-a746-3e5d271642dd None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:27.045128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 87/175] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:27.046414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0e0ef-dc2a-4228-a746-3e5d271642dd None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:27.187841 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0e0ef-dc2a-4228-a746-3e5d271642dd None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:27.209188 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49978e-7e95-49bd-a793-0f13b5c8367d None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:27.246528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:27.263772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335b8ce-9611-46fe-8eea-4c4a29ae801d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:27.264416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335b8ce-9611-46fe-8eea-4c4a29ae801d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:27.264696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335b8ce-9611-46fe-8eea-4c4a29ae801d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:27.264970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335b8ce-9611-46fe-8eea-4c4a29ae801d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:27.265921 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335b8ce-9611-46fe-8eea-4c4a29ae801d None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa, user.id=5c92b48d12534a97833b0661f5d186ee) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:27.266294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335b8ce-9611-46fe-8eea-4c4a29ae801d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:27.267820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335b8ce-9611-46fe-8eea-4c4a29ae801d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:27.283969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 88/176] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:27 2022] GET /identity/v3/role_assignments?user.id=5c92b48d12534a97833b0661f5d186ee&scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:27.290567 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335b8ce-9611-46fe-8eea-4c4a29ae801d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:27.303678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936d20b-53db-4177-a426-1fa5ab9a8245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:27.304352 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936d20b-53db-4177-a426-1fa5ab9a8245 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:27.304630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936d20b-53db-4177-a426-1fa5ab9a8245 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:27.304901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936d20b-53db-4177-a426-1fa5ab9a8245 None admin] PATH_INFO: `/v3/users/5d6589676dfc4249ad16e0ef65b56ddb` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:27.309759 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936d20b-53db-4177-a426-1fa5ab9a8245 None admin] RBAC: Authorizing `identity:get_user(user_id=5d6589676dfc4249ad16e0ef65b56ddb)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:27.311322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936d20b-53db-4177-a426-1fa5ab9a8245 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:27.313143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 89/177] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:26 2022] GET /identity/v3/users/5d6589676dfc4249ad16e0ef65b56ddb => generated 289 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:27.323113 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62187a34-cace-447e-90a3-53b42f7d0111 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:27.323464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62187a34-cace-447e-90a3-53b42f7d0111 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:27.323660 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62187a34-cace-447e-90a3-53b42f7d0111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:27.365593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-62187a34-cace-447e-90a3-53b42f7d0111 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:27.366863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-62187a34-cace-447e-90a3-53b42f7d0111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:27.487212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335b8ce-9611-46fe-8eea-4c4a29ae801d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:27.493744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 90/178] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:27.494775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62187a34-cace-447e-90a3-53b42f7d0111 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:27.516658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:27.532268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def29e1-02a3-4d5c-b97f-69e4920b926b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:27.533371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def29e1-02a3-4d5c-b97f-69e4920b926b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:27.533842 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def29e1-02a3-4d5c-b97f-69e4920b926b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:27.534319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def29e1-02a3-4d5c-b97f-69e4920b926b None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:27.535517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def29e1-02a3-4d5c-b97f-69e4920b926b None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:27.537792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def29e1-02a3-4d5c-b97f-69e4920b926b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:27.541541 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 89/179] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:27 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:27.549491 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980cdb0-5174-4df0-9cf8-a6c119068df2 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:27.549491 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980cdb0-5174-4df0-9cf8-a6c119068df2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:27.549491 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980cdb0-5174-4df0-9cf8-a6c119068df2 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:27.549491 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 90/180] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:27.555058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ba3c9-02e9-47cf-b593-2ecc9ecde8e9 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:27.555227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ba3c9-02e9-47cf-b593-2ecc9ecde8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:27.555463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ba3c9-02e9-47cf-b593-2ecc9ecde8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:27.578097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-264ba3c9-02e9-47cf-b593-2ecc9ecde8e9 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:27.579482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-264ba3c9-02e9-47cf-b593-2ecc9ecde8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:27.667014 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62187a34-cace-447e-90a3-53b42f7d0111 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:27.676702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:27.691506 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 91/181] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:27.698865 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ba3c9-02e9-47cf-b593-2ecc9ecde8e9 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:27.715768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295a9ac-a967-45d4-9f9c-b0f63dba5d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:27.717173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295a9ac-a967-45d4-9f9c-b0f63dba5d43 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:27.717636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295a9ac-a967-45d4-9f9c-b0f63dba5d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:27.718407 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295a9ac-a967-45d4-9f9c-b0f63dba5d43 None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:27.720132 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295a9ac-a967-45d4-9f9c-b0f63dba5d43 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:27.721944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295a9ac-a967-45d4-9f9c-b0f63dba5d43 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:27.724452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 91/182] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:27 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:27.731426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295a9ac-a967-45d4-9f9c-b0f63dba5d43 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:27.825627 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:27.855398 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ba3c9-02e9-47cf-b593-2ecc9ecde8e9 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:27.868073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295a9ac-a967-45d4-9f9c-b0f63dba5d43 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:27.900835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd5075-62a7-4ef9-968f-f2657561c49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:27.901537 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd5075-62a7-4ef9-968f-f2657561c49d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:27.901717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd5075-62a7-4ef9-968f-f2657561c49d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:27.901956 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd5075-62a7-4ef9-968f-f2657561c49d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:27.902717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd5075-62a7-4ef9-968f-f2657561c49d None admin] RBAC: Adding query filter params (role.id=d43148a29472443ba7dfe2424ec60baf, scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa, user.id=b7b49d2a1ef74401b3242303d28a3f46) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:27.903031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd5075-62a7-4ef9-968f-f2657561c49d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:27.904464 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd5075-62a7-4ef9-968f-f2657561c49d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:27.906417 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08050b-9d71-4fa8-9eca-76ec8f377ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:27.906938 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08050b-9d71-4fa8-9eca-76ec8f377ee9 None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:27.907219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08050b-9d71-4fa8-9eca-76ec8f377ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:27.907494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08050b-9d71-4fa8-9eca-76ec8f377ee9 None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa/users/5d6589676dfc4249ad16e0ef65b56ddb/roles/80400d1e5fdb44b3a92a468f6b62b248` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:27.909855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08050b-9d71-4fa8-9eca-76ec8f377ee9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f1f1e3af21d472cb21dc183a536cdfa, user_id=5d6589676dfc4249ad16e0ef65b56ddb, role_id=80400d1e5fdb44b3a92a468f6b62b248)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:27.911186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08050b-9d71-4fa8-9eca-76ec8f377ee9 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:27.912415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 92/183] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:27 2022] GET /identity/v3/role_assignments?user.id=b7b49d2a1ef74401b3242303d28a3f46&scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa&role.id=d43148a29472443ba7dfe2424ec60baf => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:27.913531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d7169-e69d-4fa0-83c9-9034d5a58cec None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:27.913912 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d7169-e69d-4fa0-83c9-9034d5a58cec None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:27.914079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d7169-e69d-4fa0-83c9-9034d5a58cec None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:27.916026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 93/184] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:27.920656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d7169-e69d-4fa0-83c9-9034d5a58cec None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:27.935519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 92/185] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:52:27 2022] PUT /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa/users/5d6589676dfc4249ad16e0ef65b56ddb/roles/80400d1e5fdb44b3a92a468f6b62b248 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:52:27.937398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bddf95-f0a3-487b-abea-cc4fb1477fb3 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:27.937998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bddf95-f0a3-487b-abea-cc4fb1477fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:27.938582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bddf95-f0a3-487b-abea-cc4fb1477fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:27.968940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-93bddf95-f0a3-487b-abea-cc4fb1477fb3 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:27.971336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-93bddf95-f0a3-487b-abea-cc4fb1477fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:28.074971 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 93/186] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:28.077401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:28.082885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bddf95-f0a3-487b-abea-cc4fb1477fb3 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:28.129782 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d7169-e69d-4fa0-83c9-9034d5a58cec None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:28.194089 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:28.197964 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1497d-b08f-44ef-8638-195bb8422005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:28.199066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1497d-b08f-44ef-8638-195bb8422005 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:28.199342 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1497d-b08f-44ef-8638-195bb8422005 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:28.199735 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1497d-b08f-44ef-8638-195bb8422005 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:28.213202 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1497d-b08f-44ef-8638-195bb8422005 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-9fa1497d-b08f-44ef-8638-195bb8422005 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:28.218536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:28.219462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:28.220268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:28.220268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:28.220268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:28.220268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:28.220268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:28.220268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:28.220268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:28.220268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:28.220268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:28.220268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:28.220268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:28.220268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:28.220268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 94/187] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:52:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:28.227563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1497d-b08f-44ef-8638-195bb8422005 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:28.269055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bddf95-f0a3-487b-abea-cc4fb1477fb3 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:28.316110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb27295-07d6-4c52-b3fd-2a9a947d62a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:28.316613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb27295-07d6-4c52-b3fd-2a9a947d62a5 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:28.316887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb27295-07d6-4c52-b3fd-2a9a947d62a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:28.317151 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb27295-07d6-4c52-b3fd-2a9a947d62a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:28.335432 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb27295-07d6-4c52-b3fd-2a9a947d62a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-cdb27295-07d6-4c52-b3fd-2a9a947d62a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:28.342484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:28.343392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:28.344161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:28.344161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:28.344161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:28.344161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:28.344161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:28.344161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:28.344161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:28.344161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:28.344161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:28.344161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:28.344161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:28.344161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:28.346693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 94/188] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:52:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:28.354564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb27295-07d6-4c52-b3fd-2a9a947d62a5 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:28.366095 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1497d-b08f-44ef-8638-195bb8422005 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:28.385914 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:28.419032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e576a70-0683-464a-8c49-0d204ab666d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:28.419656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e576a70-0683-464a-8c49-0d204ab666d2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:28.419938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e576a70-0683-464a-8c49-0d204ab666d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:28.420214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e576a70-0683-464a-8c49-0d204ab666d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:28.420893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e576a70-0683-464a-8c49-0d204ab666d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:28.421208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e576a70-0683-464a-8c49-0d204ab666d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:28.422622 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e576a70-0683-464a-8c49-0d204ab666d2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:28.431679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 95/189] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:28.436439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e576a70-0683-464a-8c49-0d204ab666d2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:28.489215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb27295-07d6-4c52-b3fd-2a9a947d62a5 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:28.512965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:28.534497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c551e-cc99-4658-aec1-a17295f229de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:28.535337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c551e-cc99-4658-aec1-a17295f229de None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:28.535621 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c551e-cc99-4658-aec1-a17295f229de None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:28.535889 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c551e-cc99-4658-aec1-a17295f229de None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:28.536579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c551e-cc99-4658-aec1-a17295f229de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:28.536884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c551e-cc99-4658-aec1-a17295f229de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:28.538598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c551e-cc99-4658-aec1-a17295f229de None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:28.547999 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e576a70-0683-464a-8c49-0d204ab666d2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:28.548252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 95/190] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:52:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:28.553852 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c551e-cc99-4658-aec1-a17295f229de None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:28.584415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e198d-fd03-4e9f-876f-fceef0eb8f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:28.585044 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e198d-fd03-4e9f-876f-fceef0eb8f2c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:28.585320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e198d-fd03-4e9f-876f-fceef0eb8f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:28.585630 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e198d-fd03-4e9f-876f-fceef0eb8f2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:28.586703 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e198d-fd03-4e9f-876f-fceef0eb8f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:28.588683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e198d-fd03-4e9f-876f-fceef0eb8f2c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:28.590279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 96/191] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:52:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:28.591118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e198d-fd03-4e9f-876f-fceef0eb8f2c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:28.646535 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:28.665650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c551e-cc99-4658-aec1-a17295f229de None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:28.702046 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c7064-51ee-4e11-86b2-ef48685000c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:28.702277 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e198d-fd03-4e9f-876f-fceef0eb8f2c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:28.702710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c7064-51ee-4e11-86b2-ef48685000c0 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:28.702987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c7064-51ee-4e11-86b2-ef48685000c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:28.703257 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c7064-51ee-4e11-86b2-ef48685000c0 None admin] PATH_INFO: `/v3/users/ef0866e48d07470db813ba0449c751f9` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:28.718427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c7064-51ee-4e11-86b2-ef48685000c0 None admin] RBAC: Authorizing `identity:get_user(user_id=ef0866e48d07470db813ba0449c751f9)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:28.720139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c7064-51ee-4e11-86b2-ef48685000c0 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:28.726277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 96/192] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:28 2022] GET /identity/v3/users/ef0866e48d07470db813ba0449c751f9 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:28.726277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c7064-51ee-4e11-86b2-ef48685000c0 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:28.743008 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17286282-4dbc-48ed-93a5-4453d34a58ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:28.744460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17286282-4dbc-48ed-93a5-4453d34a58ab None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:28.744736 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17286282-4dbc-48ed-93a5-4453d34a58ab None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:28.744989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17286282-4dbc-48ed-93a5-4453d34a58ab None admin] PATH_INFO: `/v3/users/ef0866e48d07470db813ba0449c751f9` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:28.746165 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17286282-4dbc-48ed-93a5-4453d34a58ab None admin] RBAC: Authorizing `identity:get_user(user_id=ef0866e48d07470db813ba0449c751f9)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:28.747594 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17286282-4dbc-48ed-93a5-4453d34a58ab None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:28.749340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 97/193] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:28 2022] GET /identity/v3/users/ef0866e48d07470db813ba0449c751f9 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:28.750132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17286282-4dbc-48ed-93a5-4453d34a58ab None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:28.773569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:28.846010 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c7064-51ee-4e11-86b2-ef48685000c0 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:28.859800 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17286282-4dbc-48ed-93a5-4453d34a58ab None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:28.873810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:28.885031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2174cf6-492b-4db0-94f7-739b1823d709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:28.885955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2174cf6-492b-4db0-94f7-739b1823d709 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:28.885955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2174cf6-492b-4db0-94f7-739b1823d709 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:28.886270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2174cf6-492b-4db0-94f7-739b1823d709 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:28.886938 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2174cf6-492b-4db0-94f7-739b1823d709 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:28.888500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2174cf6-492b-4db0-94f7-739b1823d709 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:28.897307 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646694bd-2c8a-431a-af44-a5de75562750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:28.898066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646694bd-2c8a-431a-af44-a5de75562750 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:28.898337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646694bd-2c8a-431a-af44-a5de75562750 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:28.898614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646694bd-2c8a-431a-af44-a5de75562750 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:28.913629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 97/194] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:52:28 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:52:28.915659 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2174cf6-492b-4db0-94f7-739b1823d709 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:28.984252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646694bd-2c8a-431a-af44-a5de75562750 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:28.986141 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646694bd-2c8a-431a-af44-a5de75562750 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:29.079588 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:29.089772 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2174cf6-492b-4db0-94f7-739b1823d709 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:29.110781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 98/195] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:29.115910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646694bd-2c8a-431a-af44-a5de75562750 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:29.137011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:29.138978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82974298-6aa2-4b6c-9765-1fa8c6146aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:29.139719 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82974298-6aa2-4b6c-9765-1fa8c6146aac None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:29.139999 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82974298-6aa2-4b6c-9765-1fa8c6146aac None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:29.140288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82974298-6aa2-4b6c-9765-1fa8c6146aac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:29.236984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82974298-6aa2-4b6c-9765-1fa8c6146aac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:29.239362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82974298-6aa2-4b6c-9765-1fa8c6146aac None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:29.242708 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646694bd-2c8a-431a-af44-a5de75562750 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:29.288062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629924e-6097-45ec-9efe-3ec572705a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:29.288716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629924e-6097-45ec-9efe-3ec572705a67 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:29.288991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629924e-6097-45ec-9efe-3ec572705a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:29.289260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629924e-6097-45ec-9efe-3ec572705a67 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:29.291108 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629924e-6097-45ec-9efe-3ec572705a67 None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:29.293130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629924e-6097-45ec-9efe-3ec572705a67 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:29.295472 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 99/196] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:29 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:29.302153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629924e-6097-45ec-9efe-3ec572705a67 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:29.370563 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:29.380423 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 98/197] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:29.386957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82974298-6aa2-4b6c-9765-1fa8c6146aac None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:29.404357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:29.427710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629924e-6097-45ec-9efe-3ec572705a67 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:29.467765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc4e18-4354-440c-898b-97a4cceebe4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:29.468495 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc4e18-4354-440c-898b-97a4cceebe4d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:29.468768 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc4e18-4354-440c-898b-97a4cceebe4d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:29.469036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc4e18-4354-440c-898b-97a4cceebe4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:29.469767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc4e18-4354-440c-898b-97a4cceebe4d None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=3a96e4f9e7134925af6e684f79748f71, user.id=ef0866e48d07470db813ba0449c751f9) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:29.470071 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc4e18-4354-440c-898b-97a4cceebe4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:29.472693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc4e18-4354-440c-898b-97a4cceebe4d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:29.479977 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 100/198] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:29 2022] GET /identity/v3/role_assignments?user.id=ef0866e48d07470db813ba0449c751f9&scope.project.id=3a96e4f9e7134925af6e684f79748f71&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:29.510439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82974298-6aa2-4b6c-9765-1fa8c6146aac None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:29.558425 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f8d96-5776-489f-8bb2-752b9a44346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:29.559063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f8d96-5776-489f-8bb2-752b9a44346b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:29.559337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f8d96-5776-489f-8bb2-752b9a44346b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:29.559607 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f8d96-5776-489f-8bb2-752b9a44346b None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:29.560932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f8d96-5776-489f-8bb2-752b9a44346b None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:29.562564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f8d96-5776-489f-8bb2-752b9a44346b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:29.564690 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 99/199] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:29 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:29.570627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc4e18-4354-440c-898b-97a4cceebe4d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:29.689210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc4e18-4354-440c-898b-97a4cceebe4d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:29.732843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a2a2d-25ea-4b13-8d52-023196e12606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:29.733733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a2a2d-25ea-4b13-8d52-023196e12606 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:29.734012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a2a2d-25ea-4b13-8d52-023196e12606 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:29.734311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a2a2d-25ea-4b13-8d52-023196e12606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:29.735123 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a2a2d-25ea-4b13-8d52-023196e12606 None admin] RBAC: Adding query filter params (role.id=d43148a29472443ba7dfe2424ec60baf, scope.project.id=3a96e4f9e7134925af6e684f79748f71, user.id=ef0866e48d07470db813ba0449c751f9) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:29.735526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a2a2d-25ea-4b13-8d52-023196e12606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:29.737212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a2a2d-25ea-4b13-8d52-023196e12606 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:29.746353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 101/200] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:29 2022] GET /identity/v3/role_assignments?user.id=ef0866e48d07470db813ba0449c751f9&scope.project.id=3a96e4f9e7134925af6e684f79748f71&role.id=d43148a29472443ba7dfe2424ec60baf => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:30.882014 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341d44b-ccd3-4406-86bb-4ef29df27ab9 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:30.882584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341d44b-ccd3-4406-86bb-4ef29df27ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:30.883018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341d44b-ccd3-4406-86bb-4ef29df27ab9 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:30.885950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 100/201] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:30.895016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0177209-0c1d-49d9-978b-2e87dd4c9302 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:30.895493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0177209-0c1d-49d9-978b-2e87dd4c9302 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:30.895989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0177209-0c1d-49d9-978b-2e87dd4c9302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:30.926783 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b0177209-0c1d-49d9-978b-2e87dd4c9302 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:30.928826 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b0177209-0c1d-49d9-978b-2e87dd4c9302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:30.932885 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:31.063960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 102/202] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:31.556536 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341d44b-ccd3-4406-86bb-4ef29df27ab9 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:31.706342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:31.807315 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341d44b-ccd3-4406-86bb-4ef29df27ab9 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:31.865458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a1551-dc80-477a-99d8-1ba49d614706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:31.866513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a1551-dc80-477a-99d8-1ba49d614706 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:31.866970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a1551-dc80-477a-99d8-1ba49d614706 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:31.867404 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a1551-dc80-477a-99d8-1ba49d614706 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:31.886787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a1551-dc80-477a-99d8-1ba49d614706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-486a1551-dc80-477a-99d8-1ba49d614706 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:31.896347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:31.897275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:31.898059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:31.898059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:31.898059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:31.898059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:31.898059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:31.898059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:31.898059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:31.898059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:31.898059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:31.898059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:31.898059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:31.898059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:31.898691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a72dd-7866-4d1a-8604-4dad9f16d6fe None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:31.898691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a72dd-7866-4d1a-8604-4dad9f16d6fe None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:31.898691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a72dd-7866-4d1a-8604-4dad9f16d6fe None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:31.899199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 101/203] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:31.900507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 103/204] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:52:31.906218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd790ea-80e9-4bf1-8786-1421b29a258c None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:31.906716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd790ea-80e9-4bf1-8786-1421b29a258c None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:31.907210 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd790ea-80e9-4bf1-8786-1421b29a258c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:31.909528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a72dd-7866-4d1a-8604-4dad9f16d6fe None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:31.951199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-ccd790ea-80e9-4bf1-8786-1421b29a258c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:31.953175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ccd790ea-80e9-4bf1-8786-1421b29a258c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:32.114482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a72dd-7866-4d1a-8604-4dad9f16d6fe None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:32.129743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 102/205] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:32.155503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffaf23-2e0f-4291-a400-dc3412ffaa33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:32.156124 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffaf23-2e0f-4291-a400-dc3412ffaa33 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:32.156399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffaf23-2e0f-4291-a400-dc3412ffaa33 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:32.156669 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffaf23-2e0f-4291-a400-dc3412ffaa33 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:32.157395 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffaf23-2e0f-4291-a400-dc3412ffaa33 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:32.157696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffaf23-2e0f-4291-a400-dc3412ffaa33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:32.159284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffaf23-2e0f-4291-a400-dc3412ffaa33 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:32.166461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:32.172324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 104/206] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:32.178877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd790ea-80e9-4bf1-8786-1421b29a258c None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:32.259425 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:32.317322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd790ea-80e9-4bf1-8786-1421b29a258c None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:32.359373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5af385-08db-4a14-a7f7-5a292c57e33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:32.360007 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5af385-08db-4a14-a7f7-5a292c57e33a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:32.360285 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5af385-08db-4a14-a7f7-5a292c57e33a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:32.360553 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5af385-08db-4a14-a7f7-5a292c57e33a None admin] PATH_INFO: `/v3/users/5d6589676dfc4249ad16e0ef65b56ddb` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:32.361687 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5af385-08db-4a14-a7f7-5a292c57e33a None admin] RBAC: Authorizing `identity:get_user(user_id=5d6589676dfc4249ad16e0ef65b56ddb)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:32.363172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5af385-08db-4a14-a7f7-5a292c57e33a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:32.364877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 103/207] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:32 2022] GET /identity/v3/users/5d6589676dfc4249ad16e0ef65b56ddb => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:32.371159 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffaf23-2e0f-4291-a400-dc3412ffaa33 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:32.474366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5af385-08db-4a14-a7f7-5a292c57e33a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:32.497093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffaf23-2e0f-4291-a400-dc3412ffaa33 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:32.540903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f989b-ada1-41b4-8891-05a6f817339c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:32.541529 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f989b-ada1-41b4-8891-05a6f817339c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:32.541807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f989b-ada1-41b4-8891-05a6f817339c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:32.542079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f989b-ada1-41b4-8891-05a6f817339c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:32.607188 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5af385-08db-4a14-a7f7-5a292c57e33a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:32.626533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:32.630401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f989b-ada1-41b4-8891-05a6f817339c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:32.632369 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f989b-ada1-41b4-8891-05a6f817339c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:32.635622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:32.650920 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed1207-0597-4b89-aef7-eb5ee5206c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:32.651554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed1207-0597-4b89-aef7-eb5ee5206c63 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:32.651845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed1207-0597-4b89-aef7-eb5ee5206c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:32.652093 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed1207-0597-4b89-aef7-eb5ee5206c63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:32.664785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed1207-0597-4b89-aef7-eb5ee5206c63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-a8ed1207-0597-4b89-aef7-eb5ee5206c63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:32.671497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:32.674028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:32.674866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:32.674866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:32.674866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:32.674866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:32.674866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:32.674866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:32.674866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:32.674866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:32.674866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:32.674866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:32.674866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:32.674866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:32.674866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 104/208] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:52:32.674866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2ca64-8bf1-464e-86ad-363b51acca45 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:32.675581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2ca64-8bf1-464e-86ad-363b51acca45 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:32.675581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2ca64-8bf1-464e-86ad-363b51acca45 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:32.675581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 105/209] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:32.676806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f62b5-bcde-4394-94ac-85ddd4aba846 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:32.676806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f62b5-bcde-4394-94ac-85ddd4aba846 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:32.677142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f62b5-bcde-4394-94ac-85ddd4aba846 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:32.682968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 106/210] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:32.683677 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f62b5-bcde-4394-94ac-85ddd4aba846 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:32.750816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 105/211] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:32.752017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e2bf0-a84b-4679-816d-20296587488b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:32.752298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e2bf0-a84b-4679-816d-20296587488b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:32.752565 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e2bf0-a84b-4679-816d-20296587488b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:32.771388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-823e2bf0-a84b-4679-816d-20296587488b None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:32.772565 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-823e2bf0-a84b-4679-816d-20296587488b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:32.799478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f62b5-bcde-4394-94ac-85ddd4aba846 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:32.835512 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e17fbc-8d53-4c2e-9e8e-6f51c3bef2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:32.836119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e17fbc-8d53-4c2e-9e8e-6f51c3bef2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:32.836396 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e17fbc-8d53-4c2e-9e8e-6f51c3bef2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:32.836660 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e17fbc-8d53-4c2e-9e8e-6f51c3bef2f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:32.837358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e17fbc-8d53-4c2e-9e8e-6f51c3bef2f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:32.837659 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e17fbc-8d53-4c2e-9e8e-6f51c3bef2f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:32.839091 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e17fbc-8d53-4c2e-9e8e-6f51c3bef2f0 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:32.845029 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 106/212] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:32.846517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbe7a4-d79c-4288-b70e-eb0926052feb None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:32.846805 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbe7a4-d79c-4288-b70e-eb0926052feb None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:32.847080 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbe7a4-d79c-4288-b70e-eb0926052feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:32.859826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 107/213] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:32.860922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e17fbc-8d53-4c2e-9e8e-6f51c3bef2f0 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:32.871128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-56dbe7a4-d79c-4288-b70e-eb0926052feb None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:32.872420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-56dbe7a4-d79c-4288-b70e-eb0926052feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:32.903812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:32.923669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:32.961542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 107/214] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:32.962816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbe7a4-d79c-4288-b70e-eb0926052feb None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:32.989859 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e17fbc-8d53-4c2e-9e8e-6f51c3bef2f0 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:33.027558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85446f-7d29-46af-9122-0e18ecc34ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:33.028294 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85446f-7d29-46af-9122-0e18ecc34ada None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:33.028647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85446f-7d29-46af-9122-0e18ecc34ada None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:33.029019 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85446f-7d29-46af-9122-0e18ecc34ada None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:33.030317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85446f-7d29-46af-9122-0e18ecc34ada None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:33.031897 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85446f-7d29-46af-9122-0e18ecc34ada None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:33.033838 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 108/215] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:32 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:33.041523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85446f-7d29-46af-9122-0e18ecc34ada None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:33.102104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbe7a4-d79c-4288-b70e-eb0926052feb None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:33.145672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e2feb-7bc2-4a7b-981b-b2aeaac0fe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:33.146619 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e2feb-7bc2-4a7b-981b-b2aeaac0fe13 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:33.147088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e2feb-7bc2-4a7b-981b-b2aeaac0fe13 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:33.147502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e2feb-7bc2-4a7b-981b-b2aeaac0fe13 None admin] PATH_INFO: `/v3/users/9cc48f17f5f246d8a9eeed99ff863740` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:33.156035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85446f-7d29-46af-9122-0e18ecc34ada None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:33.164371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e2feb-7bc2-4a7b-981b-b2aeaac0fe13 None admin] RBAC: Authorizing `identity:get_user(user_id=9cc48f17f5f246d8a9eeed99ff863740)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:33.167051 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e2feb-7bc2-4a7b-981b-b2aeaac0fe13 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:33.169444 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 108/216] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:32 2022] GET /identity/v3/users/9cc48f17f5f246d8a9eeed99ff863740 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:33.170593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e2feb-7bc2-4a7b-981b-b2aeaac0fe13 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:33.176287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:33.193277 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:33.211445 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43532b6c-a73e-4a21-9306-50e6d95e484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:33.212768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43532b6c-a73e-4a21-9306-50e6d95e484f None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:33.213141 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43532b6c-a73e-4a21-9306-50e6d95e484f None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:33.213562 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43532b6c-a73e-4a21-9306-50e6d95e484f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:33.214652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43532b6c-a73e-4a21-9306-50e6d95e484f None admin] RBAC: Adding query filter params (role.id=80400d1e5fdb44b3a92a468f6b62b248, scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa, user.id=5d6589676dfc4249ad16e0ef65b56ddb) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:33.215326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43532b6c-a73e-4a21-9306-50e6d95e484f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:33.218005 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43532b6c-a73e-4a21-9306-50e6d95e484f None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:33.224896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 109/217] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:33 2022] GET /identity/v3/role_assignments?user.id=5d6589676dfc4249ad16e0ef65b56ddb&scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa&role.id=80400d1e5fdb44b3a92a468f6b62b248 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:33.225796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43532b6c-a73e-4a21-9306-50e6d95e484f None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:33.302704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e2feb-7bc2-4a7b-981b-b2aeaac0fe13 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:33.341827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54c93d-c8b8-45ae-92f0-1d3080fa8d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:33.342784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54c93d-c8b8-45ae-92f0-1d3080fa8d5b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:33.343065 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54c93d-c8b8-45ae-92f0-1d3080fa8d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:33.343330 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54c93d-c8b8-45ae-92f0-1d3080fa8d5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:33.357109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54c93d-c8b8-45ae-92f0-1d3080fa8d5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:33.360907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43532b6c-a73e-4a21-9306-50e6d95e484f None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-7d54c93d-c8b8-45ae-92f0-1d3080fa8d5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:33.364271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:33.365197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:33.365987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:33.365987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:33.365987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:33.365987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:33.365987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:33.365987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:33.365987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:33.365987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:33.365987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:33.365987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:33.365987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:33.365987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:33.367642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 109/218] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:52:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:33.371591 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13199f2-813b-404c-881e-5b5e58b7ca2c None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:33.372802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13199f2-813b-404c-881e-5b5e58b7ca2c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:33.373219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13199f2-813b-404c-881e-5b5e58b7ca2c None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:33.374721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 110/219] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:33.379876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13199f2-813b-404c-881e-5b5e58b7ca2c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:33.408476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce9a22-e33f-4b07-b00c-03deb40e157c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:33.409355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce9a22-e33f-4b07-b00c-03deb40e157c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:33.409801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce9a22-e33f-4b07-b00c-03deb40e157c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:33.410296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce9a22-e33f-4b07-b00c-03deb40e157c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:33.470645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:33.490411 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:33.504335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce9a22-e33f-4b07-b00c-03deb40e157c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:33.505809 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce9a22-e33f-4b07-b00c-03deb40e157c None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:33.515019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13199f2-813b-404c-881e-5b5e58b7ca2c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:33.549898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96b971-d5ab-430d-9e69-b8f6c21541fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:33.550546 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96b971-d5ab-430d-9e69-b8f6c21541fe None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:33.550823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96b971-d5ab-430d-9e69-b8f6c21541fe None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:33.551082 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96b971-d5ab-430d-9e69-b8f6c21541fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:33.552453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96b971-d5ab-430d-9e69-b8f6c21541fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:33.553882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96b971-d5ab-430d-9e69-b8f6c21541fe None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:33.555821 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 111/220] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:52:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:33.558275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96b971-d5ab-430d-9e69-b8f6c21541fe None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:33.609543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 110/221] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:33.610831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c516c-288f-4a36-bc9e-d0e6101a952e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:33.611115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c516c-288f-4a36-bc9e-d0e6101a952e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:33.611386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c516c-288f-4a36-bc9e-d0e6101a952e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:33.630030 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-ed1c516c-288f-4a36-bc9e-d0e6101a952e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:33.631244 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ed1c516c-288f-4a36-bc9e-d0e6101a952e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:33.658948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96b971-d5ab-430d-9e69-b8f6c21541fe None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:33.696073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6d82b-c25a-4b4e-94a5-5d66f4b7f1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:33.697272 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6d82b-c25a-4b4e-94a5-5d66f4b7f1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:33.697549 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6d82b-c25a-4b4e-94a5-5d66f4b7f1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:33.697818 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6d82b-c25a-4b4e-94a5-5d66f4b7f1e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:33.698531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6d82b-c25a-4b4e-94a5-5d66f4b7f1e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:33.698834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6d82b-c25a-4b4e-94a5-5d66f4b7f1e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:33.700901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6d82b-c25a-4b4e-94a5-5d66f4b7f1e0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:33.702445 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 111/222] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:33.703280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c516c-288f-4a36-bc9e-d0e6101a952e None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:33.712749 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 112/223] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:33.713583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6d82b-c25a-4b4e-94a5-5d66f4b7f1e0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:33.722664 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:33.725806 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:33.819931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c516c-288f-4a36-bc9e-d0e6101a952e None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:33.825660 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6d82b-c25a-4b4e-94a5-5d66f4b7f1e0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:33.853985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82140deb-a9f4-422e-8fe4-b4f8e3f3daf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:33.854632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82140deb-a9f4-422e-8fe4-b4f8e3f3daf5 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:33.854910 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82140deb-a9f4-422e-8fe4-b4f8e3f3daf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:33.855177 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82140deb-a9f4-422e-8fe4-b4f8e3f3daf5 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:33.855822 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82140deb-a9f4-422e-8fe4-b4f8e3f3daf5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:33.857264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82140deb-a9f4-422e-8fe4-b4f8e3f3daf5 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:33.863029 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19ea3b-2847-455b-a7a1-b55fa04f9846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:33.863827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19ea3b-2847-455b-a7a1-b55fa04f9846 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:33.864266 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19ea3b-2847-455b-a7a1-b55fa04f9846 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:33.864694 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19ea3b-2847-455b-a7a1-b55fa04f9846 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:33.866090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19ea3b-2847-455b-a7a1-b55fa04f9846 None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:33.867843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19ea3b-2847-455b-a7a1-b55fa04f9846 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:33.870568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 113/224] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:33 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:33.872001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19ea3b-2847-455b-a7a1-b55fa04f9846 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:33.894148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 112/225] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:52:33 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:52:33.895459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82140deb-a9f4-422e-8fe4-b4f8e3f3daf5 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:33.972454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:33.975275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:34.009729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19ea3b-2847-455b-a7a1-b55fa04f9846 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:34.022602 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82140deb-a9f4-422e-8fe4-b4f8e3f3daf5 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:34.052131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efef391-9537-48a9-ba11-4ad50d062a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:34.052778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efef391-9537-48a9-ba11-4ad50d062a40 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:34.053053 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efef391-9537-48a9-ba11-4ad50d062a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:34.053318 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efef391-9537-48a9-ba11-4ad50d062a40 None admin] PATH_INFO: `/v3/users/ef0866e48d07470db813ba0449c751f9` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:34.054528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efef391-9537-48a9-ba11-4ad50d062a40 None admin] RBAC: Authorizing `identity:get_user(user_id=ef0866e48d07470db813ba0449c751f9)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:34.056077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efef391-9537-48a9-ba11-4ad50d062a40 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:34.058129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 114/226] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:33 2022] GET /identity/v3/users/ef0866e48d07470db813ba0449c751f9 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:34.058949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efef391-9537-48a9-ba11-4ad50d062a40 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:34.073984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149ebb8-6614-4dd4-8c32-608d4c23931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:34.074626 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149ebb8-6614-4dd4-8c32-608d4c23931a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:34.074899 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149ebb8-6614-4dd4-8c32-608d4c23931a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:34.075164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149ebb8-6614-4dd4-8c32-608d4c23931a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:34.075894 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149ebb8-6614-4dd4-8c32-608d4c23931a None admin] RBAC: Adding query filter params (role.id=e385f7d11fac4932a910ec0ef95e2a43, scope.project.id=3a96e4f9e7134925af6e684f79748f71, user.id=9cc48f17f5f246d8a9eeed99ff863740) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:34.076200 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149ebb8-6614-4dd4-8c32-608d4c23931a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:34.077706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149ebb8-6614-4dd4-8c32-608d4c23931a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:34.083295 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 113/227] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:33 2022] GET /identity/v3/role_assignments?user.id=9cc48f17f5f246d8a9eeed99ff863740&scope.project.id=3a96e4f9e7134925af6e684f79748f71&role.id=e385f7d11fac4932a910ec0ef95e2a43 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:34.084099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149ebb8-6614-4dd4-8c32-608d4c23931a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:34.217207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efef391-9537-48a9-ba11-4ad50d062a40 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:34.241433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149ebb8-6614-4dd4-8c32-608d4c23931a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:34.264680 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e1256-5697-4e42-92c9-669c6cacacf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:34.265350 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e1256-5697-4e42-92c9-669c6cacacf5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:34.265628 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e1256-5697-4e42-92c9-669c6cacacf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:34.265900 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e1256-5697-4e42-92c9-669c6cacacf5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:34.281224 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:34.281591 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e1256-5697-4e42-92c9-669c6cacacf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-794e1256-5697-4e42-92c9-669c6cacacf5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:34.288447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:34.289378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:34.290241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:34.290241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:34.290241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:34.290241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:34.290241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:34.290241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:34.290241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:34.290241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:34.290241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:34.290241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:34.290241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:34.290241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:34.290241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 115/228] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:52:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:34.298166 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2eede1-86af-4bbc-98b5-af89a40b6345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:34.298961 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2eede1-86af-4bbc-98b5-af89a40b6345 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:34.299066 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2eede1-86af-4bbc-98b5-af89a40b6345 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:34.299726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2eede1-86af-4bbc-98b5-af89a40b6345 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:34.300846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e1256-5697-4e42-92c9-669c6cacacf5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:34.301498 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2eede1-86af-4bbc-98b5-af89a40b6345 None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:34.303107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2eede1-86af-4bbc-98b5-af89a40b6345 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:34.305375 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 114/229] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:34 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:34.306425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:34.311802 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2eede1-86af-4bbc-98b5-af89a40b6345 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:34.439586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e1256-5697-4e42-92c9-669c6cacacf5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:34.449221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2eede1-86af-4bbc-98b5-af89a40b6345 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:34.481627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f205f8c-d966-42ca-af0b-fca19928173b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:34.482510 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f205f8c-d966-42ca-af0b-fca19928173b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:34.482935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f205f8c-d966-42ca-af0b-fca19928173b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:34.483597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f205f8c-d966-42ca-af0b-fca19928173b None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:34.484442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f205f8c-d966-42ca-af0b-fca19928173b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:34.484892 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f205f8c-d966-42ca-af0b-fca19928173b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:34.485392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1a725-71c0-42ea-9c37-23b3c751b36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:34.486217 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1a725-71c0-42ea-9c37-23b3c751b36a None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:34.486501 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1a725-71c0-42ea-9c37-23b3c751b36a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:34.486777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1a725-71c0-42ea-9c37-23b3c751b36a None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71/users/ef0866e48d07470db813ba0449c751f9/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:34.487139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f205f8c-d966-42ca-af0b-fca19928173b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:34.488831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1a725-71c0-42ea-9c37-23b3c751b36a None admin] RBAC: Authorizing `identity:create_grant(project_id=3a96e4f9e7134925af6e684f79748f71, user_id=ef0866e48d07470db813ba0449c751f9, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:34.490326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1a725-71c0-42ea-9c37-23b3c751b36a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:34.497596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 116/230] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:52:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:34.503873 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f205f8c-d966-42ca-af0b-fca19928173b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:34.514448 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 115/231] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:52:34 2022] PUT /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71/users/ef0866e48d07470db813ba0449c751f9/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:52:34.595369 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:34.638831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f205f8c-d966-42ca-af0b-fca19928173b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:34.679476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1a133-fa93-46ff-a5f2-8512026b104f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:34.680141 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1a133-fa93-46ff-a5f2-8512026b104f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:34.680582 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1a133-fa93-46ff-a5f2-8512026b104f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:34.681433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1a133-fa93-46ff-a5f2-8512026b104f None admin] PATH_INFO: `/v3/users/ef0866e48d07470db813ba0449c751f9` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:34.682801 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1a133-fa93-46ff-a5f2-8512026b104f None admin] RBAC: Authorizing `identity:get_user(user_id=ef0866e48d07470db813ba0449c751f9)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:34.692293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1a133-fa93-46ff-a5f2-8512026b104f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:34.695488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 117/232] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:34 2022] GET /identity/v3/users/ef0866e48d07470db813ba0449c751f9 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:34.699263 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1a725-71c0-42ea-9c37-23b3c751b36a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:34.777645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:34.826865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1a725-71c0-42ea-9c37-23b3c751b36a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:34.863053 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e9e0ae-2d1d-47f4-8b18-1760bec45a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:34.863997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9e0ae-2d1d-47f4-8b18-1760bec45a46 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:34.864432 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9e0ae-2d1d-47f4-8b18-1760bec45a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:34.864846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9e0ae-2d1d-47f4-8b18-1760bec45a46 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:34.866165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e9e0ae-2d1d-47f4-8b18-1760bec45a46 None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:34.867773 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e9e0ae-2d1d-47f4-8b18-1760bec45a46 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:34.870181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 116/233] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:34 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:34.878146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1a133-fa93-46ff-a5f2-8512026b104f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:35.058352 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1a133-fa93-46ff-a5f2-8512026b104f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:35.110052 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954568e1-62d1-4280-a963-78847bdc09f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:35.110777 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954568e1-62d1-4280-a963-78847bdc09f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:35.111055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954568e1-62d1-4280-a963-78847bdc09f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:35.111329 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954568e1-62d1-4280-a963-78847bdc09f5 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71/users/ef0866e48d07470db813ba0449c751f9/roles/d43148a29472443ba7dfe2424ec60baf` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:35.113537 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954568e1-62d1-4280-a963-78847bdc09f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a96e4f9e7134925af6e684f79748f71, user_id=ef0866e48d07470db813ba0449c751f9, role_id=d43148a29472443ba7dfe2424ec60baf)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:35.115022 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954568e1-62d1-4280-a963-78847bdc09f5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:35.120852 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:35.140612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 118/234] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:52:34 2022] PUT /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71/users/ef0866e48d07470db813ba0449c751f9/roles/d43148a29472443ba7dfe2424ec60baf => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:52:36.873995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead9efa-412e-46be-b1fc-fd1279b1a95e None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:36.875047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead9efa-412e-46be-b1fc-fd1279b1a95e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:36.875548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead9efa-412e-46be-b1fc-fd1279b1a95e None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:36.877400 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 117/235] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:36.882736 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680808f-a9dd-456e-ab8c-7b31056839ae None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:36.884139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680808f-a9dd-456e-ab8c-7b31056839ae None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:36.884638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680808f-a9dd-456e-ab8c-7b31056839ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:36.911188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2680808f-a9dd-456e-ab8c-7b31056839ae None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:36.912775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2680808f-a9dd-456e-ab8c-7b31056839ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:36.934423 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707171f5-20f7-47f1-bad2-7d0a509bf77a None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:36.934773 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707171f5-20f7-47f1-bad2-7d0a509bf77a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:36.934942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707171f5-20f7-47f1-bad2-7d0a509bf77a None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:36.936551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 118/236] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:36.941716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8c187-e9e4-46e6-be2e-7bed67230112 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:36.941994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8c187-e9e4-46e6-be2e-7bed67230112 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:36.946540 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8c187-e9e4-46e6-be2e-7bed67230112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:36.965506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-cfc8c187-e9e4-46e6-be2e-7bed67230112 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:36.966964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-cfc8c187-e9e4-46e6-be2e-7bed67230112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:37.009536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 119/237] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:37.011230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7532e13-c2b7-40a2-9669-83adf016edd3 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:37.011716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7532e13-c2b7-40a2-9669-83adf016edd3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:37.012192 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7532e13-c2b7-40a2-9669-83adf016edd3 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:37.013922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 120/238] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:37.032266 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94777c6e-383f-44c5-b6c3-abc0073c5a09 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:37.032949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94777c6e-383f-44c5-b6c3-abc0073c5a09 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:37.033426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94777c6e-383f-44c5-b6c3-abc0073c5a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:37.062600 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-94777c6e-383f-44c5-b6c3-abc0073c5a09 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:37.066335 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-94777c6e-383f-44c5-b6c3-abc0073c5a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:37.094413 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 119/239] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:37.143456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 121/240] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:37.262996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172087b-af92-4945-8c68-09eaf54be87b None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:37.263611 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172087b-af92-4945-8c68-09eaf54be87b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:37.264122 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172087b-af92-4945-8c68-09eaf54be87b None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:37.265691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 120/241] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:37.271473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2d13b-252c-44eb-bd81-30c889d75ba1 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:37.271983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2d13b-252c-44eb-bd81-30c889d75ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:37.272473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2d13b-252c-44eb-bd81-30c889d75ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:37.297525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7bd2d13b-252c-44eb-bd81-30c889d75ba1 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:37.299093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7bd2d13b-252c-44eb-bd81-30c889d75ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:37.364093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:37.364991 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172087b-af92-4945-8c68-09eaf54be87b None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:37.389678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 122/242] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:37.444528 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:37.474390 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2d13b-252c-44eb-bd81-30c889d75ba1 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:37.517054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172087b-af92-4945-8c68-09eaf54be87b None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:37.568537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8af5d9-3a1e-4ce8-a31c-ce78bf5a77a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:37.569639 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8af5d9-3a1e-4ce8-a31c-ce78bf5a77a0 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:37.569920 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8af5d9-3a1e-4ce8-a31c-ce78bf5a77a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:37.570215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8af5d9-3a1e-4ce8-a31c-ce78bf5a77a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:37.572166 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8af5d9-3a1e-4ce8-a31c-ce78bf5a77a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:37.573880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8af5d9-3a1e-4ce8-a31c-ce78bf5a77a0 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:37.576509 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 121/243] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:52:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:37.577362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8af5d9-3a1e-4ce8-a31c-ce78bf5a77a0 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:37.623286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2d13b-252c-44eb-bd81-30c889d75ba1 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:37.664385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86051c20-d99c-4bd5-b212-db44d1e32e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:37.665487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86051c20-d99c-4bd5-b212-db44d1e32e70 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:37.666022 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86051c20-d99c-4bd5-b212-db44d1e32e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:37.666318 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86051c20-d99c-4bd5-b212-db44d1e32e70 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:37.679881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86051c20-d99c-4bd5-b212-db44d1e32e70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-86051c20-d99c-4bd5-b212-db44d1e32e70 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:37.687084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:37.687966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:37.688791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:37.688791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:37.688791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:37.688791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:37.688791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:37.688791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:37.688791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:37.688791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:37.688791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:37.688791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:37.688791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:37.688791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:37.689394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 123/244] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:52:37.689925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86051c20-d99c-4bd5-b212-db44d1e32e70 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:37.710034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8af5d9-3a1e-4ce8-a31c-ce78bf5a77a0 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:37.732060 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:37.747311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:37.750364 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba575e-6bb0-4b6b-841b-41259c0e993c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:37.751220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba575e-6bb0-4b6b-841b-41259c0e993c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:37.751491 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba575e-6bb0-4b6b-841b-41259c0e993c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:37.751765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba575e-6bb0-4b6b-841b-41259c0e993c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:37.763250 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba575e-6bb0-4b6b-841b-41259c0e993c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-a0ba575e-6bb0-4b6b-841b-41259c0e993c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:37.768172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:37.769061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:37.769844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:37.769844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:37.769844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:37.769844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:37.769844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:37.769844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:37.769844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:37.769844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:37.769844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:37.769844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:37.769844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:37.769844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:37.769844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 122/245] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:37.771679 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba575e-6bb0-4b6b-841b-41259c0e993c None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:37.817040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86051c20-d99c-4bd5-b212-db44d1e32e70 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:37.860226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d679b5e-9f06-4b4f-a324-4eaef3098db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:37.860777 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d679b5e-9f06-4b4f-a324-4eaef3098db2 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:37.861069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d679b5e-9f06-4b4f-a324-4eaef3098db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:37.861325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d679b5e-9f06-4b4f-a324-4eaef3098db2 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:37.861972 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d679b5e-9f06-4b4f-a324-4eaef3098db2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:37.863624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d679b5e-9f06-4b4f-a324-4eaef3098db2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:37.884803 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba575e-6bb0-4b6b-841b-41259c0e993c None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:37.895619 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 124/246] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:52:37 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:52:37.896764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d679b5e-9f06-4b4f-a324-4eaef3098db2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:37.927230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25469661-6130-4929-badd-94e0bf0b87d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:37.927870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25469661-6130-4929-badd-94e0bf0b87d2 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:37.928160 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25469661-6130-4929-badd-94e0bf0b87d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:37.928421 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25469661-6130-4929-badd-94e0bf0b87d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:37.929749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25469661-6130-4929-badd-94e0bf0b87d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:37.930250 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25469661-6130-4929-badd-94e0bf0b87d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:37.931767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25469661-6130-4929-badd-94e0bf0b87d2 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:37.943119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 123/247] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:37.943119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25469661-6130-4929-badd-94e0bf0b87d2 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:37.994295 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:38.021884 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:38.027878 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d679b5e-9f06-4b4f-a324-4eaef3098db2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:38.062345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25469661-6130-4929-badd-94e0bf0b87d2 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:38.065968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc37eff-e5c1-49de-8ea2-407d330535e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:38.066641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc37eff-e5c1-49de-8ea2-407d330535e7 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:38.066916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc37eff-e5c1-49de-8ea2-407d330535e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:38.067186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc37eff-e5c1-49de-8ea2-407d330535e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:38.080118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc37eff-e5c1-49de-8ea2-407d330535e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-2bc37eff-e5c1-49de-8ea2-407d330535e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:38.085599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:38.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:38.087908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:38.087908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:38.087908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:38.087908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:38.087908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:38.087908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:38.087908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:38.087908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:38.087908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:38.087908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:38.087908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:38.087908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:38.088527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 125/248] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:52:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:38.089381 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66831d0-19af-45a3-9af6-dbaad3b25e48 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:38.089662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66831d0-19af-45a3-9af6-dbaad3b25e48 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:38.089931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66831d0-19af-45a3-9af6-dbaad3b25e48 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:38.096073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 126/249] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:38.096073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66831d0-19af-45a3-9af6-dbaad3b25e48 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:38.109040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f18e6d-2c0f-4c2e-8b58-317271d96f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:38.109645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f18e6d-2c0f-4c2e-8b58-317271d96f67 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:38.109924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f18e6d-2c0f-4c2e-8b58-317271d96f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:38.110224 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f18e6d-2c0f-4c2e-8b58-317271d96f67 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:38.110897 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f18e6d-2c0f-4c2e-8b58-317271d96f67 None admin] RBAC: Adding query filter params (name=member) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:38.111692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f18e6d-2c0f-4c2e-8b58-317271d96f67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:38.112564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f18e6d-2c0f-4c2e-8b58-317271d96f67 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:38.121747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 124/250] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:38.122694 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f18e6d-2c0f-4c2e-8b58-317271d96f67 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:38.218957 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66831d0-19af-45a3-9af6-dbaad3b25e48 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:38.257375 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f18e6d-2c0f-4c2e-8b58-317271d96f67 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:38.261336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:38.267236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e9f0f-45fb-451f-b80f-0f85360df187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:38.267907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e9f0f-45fb-451f-b80f-0f85360df187 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:38.268213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e9f0f-45fb-451f-b80f-0f85360df187 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:38.268485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e9f0f-45fb-451f-b80f-0f85360df187 None admin] PATH_INFO: `/v3/users/f4e8b36563d1429d9d37523e65dafd7a` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:38.285388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e9f0f-45fb-451f-b80f-0f85360df187 None admin] RBAC: Authorizing `identity:get_user(user_id=f4e8b36563d1429d9d37523e65dafd7a)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:38.287154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e9f0f-45fb-451f-b80f-0f85360df187 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:38.289147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 127/251] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:38 2022] GET /identity/v3/users/f4e8b36563d1429d9d37523e65dafd7a => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:38.291355 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9358b2-c158-4aeb-995a-fa52b0a6cb1e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:38.291842 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9358b2-c158-4aeb-995a-fa52b0a6cb1e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:38.292287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9358b2-c158-4aeb-995a-fa52b0a6cb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:38.303329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432ba41-18a7-4cb4-b242-4ca48c7637d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:38.304032 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432ba41-18a7-4cb4-b242-4ca48c7637d9 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:38.304370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432ba41-18a7-4cb4-b242-4ca48c7637d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:38.304706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432ba41-18a7-4cb4-b242-4ca48c7637d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:38.305509 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432ba41-18a7-4cb4-b242-4ca48c7637d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:38.306380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432ba41-18a7-4cb4-b242-4ca48c7637d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:38.307829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432ba41-18a7-4cb4-b242-4ca48c7637d9 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:38.314304 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1a9358b2-c158-4aeb-995a-fa52b0a6cb1e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:38.315738 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1a9358b2-c158-4aeb-995a-fa52b0a6cb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:38.322140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 125/252] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:38.322140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432ba41-18a7-4cb4-b242-4ca48c7637d9 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:38.337338 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:38.401137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 128/253] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:38.401891 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9358b2-c158-4aeb-995a-fa52b0a6cb1e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:38.434984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432ba41-18a7-4cb4-b242-4ca48c7637d9 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:38.485172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33659f31-ead4-4b39-b925-6607ab95842c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:38.486303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33659f31-ead4-4b39-b925-6607ab95842c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:38.486823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33659f31-ead4-4b39-b925-6607ab95842c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:38.487355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33659f31-ead4-4b39-b925-6607ab95842c None admin] PATH_INFO: `/v3/users/9cc48f17f5f246d8a9eeed99ff863740` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:38.488726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33659f31-ead4-4b39-b925-6607ab95842c None admin] RBAC: Authorizing `identity:get_user(user_id=9cc48f17f5f246d8a9eeed99ff863740)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:38.490416 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33659f31-ead4-4b39-b925-6607ab95842c None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:38.492363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 126/254] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:38 2022] GET /identity/v3/users/9cc48f17f5f246d8a9eeed99ff863740 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:38.493897 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33659f31-ead4-4b39-b925-6607ab95842c None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:38.519681 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9358b2-c158-4aeb-995a-fa52b0a6cb1e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:38.553858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:38.554220 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9e4ad-0456-4605-924b-1392a4e1f94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:38.555532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9e4ad-0456-4605-924b-1392a4e1f94b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:38.556451 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9e4ad-0456-4605-924b-1392a4e1f94b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:38.556751 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9e4ad-0456-4605-924b-1392a4e1f94b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:38.579569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:38.628320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9e4ad-0456-4605-924b-1392a4e1f94b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:38.629778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9e4ad-0456-4605-924b-1392a4e1f94b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:38.704193 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33659f31-ead4-4b39-b925-6607ab95842c None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:38.731969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 129/255] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:38.737505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9e4ad-0456-4605-924b-1392a4e1f94b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:38.763570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7639d-0e4e-4d7e-b7ed-a576317605cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:38.764649 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7639d-0e4e-4d7e-b7ed-a576317605cc None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:38.765125 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7639d-0e4e-4d7e-b7ed-a576317605cc None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:38.765613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7639d-0e4e-4d7e-b7ed-a576317605cc None admin] PATH_INFO: `/v3/users/ef0866e48d07470db813ba0449c751f9` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:38.767171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7639d-0e4e-4d7e-b7ed-a576317605cc None admin] RBAC: Authorizing `identity:get_user(user_id=ef0866e48d07470db813ba0449c751f9)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:38.768909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7639d-0e4e-4d7e-b7ed-a576317605cc None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:38.771008 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 127/256] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:38 2022] GET /identity/v3/users/ef0866e48d07470db813ba0449c751f9 => generated 283 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:38.771976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7639d-0e4e-4d7e-b7ed-a576317605cc None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:38.790870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:38.853302 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9e4ad-0456-4605-924b-1392a4e1f94b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:38.895752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00ce3a-cdd0-4725-8a0c-f136932a991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:38.896518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00ce3a-cdd0-4725-8a0c-f136932a991a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:38.896749 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00ce3a-cdd0-4725-8a0c-f136932a991a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:38.897019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00ce3a-cdd0-4725-8a0c-f136932a991a None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:38.898290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00ce3a-cdd0-4725-8a0c-f136932a991a None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:38.898705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7639d-0e4e-4d7e-b7ed-a576317605cc None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:38.899845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00ce3a-cdd0-4725-8a0c-f136932a991a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:38.900330 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:38.903417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 130/257] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:38 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:38.904738 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00ce3a-cdd0-4725-8a0c-f136932a991a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:38.947664 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe621f88-b76a-4fc3-93db-39ab48320cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:38.949444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe621f88-b76a-4fc3-93db-39ab48320cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:38.949876 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe621f88-b76a-4fc3-93db-39ab48320cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:38.950321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe621f88-b76a-4fc3-93db-39ab48320cf4 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:38.951657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe621f88-b76a-4fc3-93db-39ab48320cf4 None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:38.953253 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe621f88-b76a-4fc3-93db-39ab48320cf4 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:38.956155 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 128/258] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:38 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:38.961304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe621f88-b76a-4fc3-93db-39ab48320cf4 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:39.023483 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00ce3a-cdd0-4725-8a0c-f136932a991a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:39.042411 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:39.062089 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b9b66-bac4-43d9-9fba-09c7f6f84cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:39.062744 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b9b66-bac4-43d9-9fba-09c7f6f84cfc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:39.063055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b9b66-bac4-43d9-9fba-09c7f6f84cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:39.063329 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b9b66-bac4-43d9-9fba-09c7f6f84cfc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:39.074982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b9b66-bac4-43d9-9fba-09c7f6f84cfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:39.080148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe621f88-b76a-4fc3-93db-39ab48320cf4 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-956b9b66-bac4-43d9-9fba-09c7f6f84cfc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:39.080528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:39.081615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:39.082802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:39.082802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:39.082802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:39.082802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:39.082802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:39.082802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:39.082802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:39.082802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:39.082802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:39.082802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:39.082802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:39.082802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:39.082802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 131/259] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:52:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:39.088294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b9b66-bac4-43d9-9fba-09c7f6f84cfc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:39.125444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff0d4e-2720-4a3f-af0b-0d321ecab3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:39.126490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff0d4e-2720-4a3f-af0b-0d321ecab3cb None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:39.127023 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff0d4e-2720-4a3f-af0b-0d321ecab3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:39.127508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff0d4e-2720-4a3f-af0b-0d321ecab3cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:39.179780 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:39.198991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b9b66-bac4-43d9-9fba-09c7f6f84cfc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:39.215879 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff0d4e-2720-4a3f-af0b-0d321ecab3cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:39.217816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff0d4e-2720-4a3f-af0b-0d321ecab3cb None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:39.237761 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8491ce0-c2f0-4042-80a9-81b2bf0853fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:39.238605 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8491ce0-c2f0-4042-80a9-81b2bf0853fc None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:39.238828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8491ce0-c2f0-4042-80a9-81b2bf0853fc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:39.239099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8491ce0-c2f0-4042-80a9-81b2bf0853fc None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71/users/9cc48f17f5f246d8a9eeed99ff863740/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:39.241292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8491ce0-c2f0-4042-80a9-81b2bf0853fc None admin] RBAC: Authorizing `identity:create_grant(project_id=3a96e4f9e7134925af6e684f79748f71, user_id=9cc48f17f5f246d8a9eeed99ff863740, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:39.242774 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8491ce0-c2f0-4042-80a9-81b2bf0853fc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:39.265137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 132/260] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:52:39 2022] PUT /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71/users/9cc48f17f5f246d8a9eeed99ff863740/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:52:39.265832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8491ce0-c2f0-4042-80a9-81b2bf0853fc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:39.355985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:39.411539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 129/261] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:39.414333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff0d4e-2720-4a3f-af0b-0d321ecab3cb None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:39.510305 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8491ce0-c2f0-4042-80a9-81b2bf0853fc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:39.557134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:39.576223 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9903ba2-e8d5-432a-9430-0a270c02be3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:39.578055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9903ba2-e8d5-432a-9430-0a270c02be3d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:39.578362 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9903ba2-e8d5-432a-9430-0a270c02be3d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:39.578638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9903ba2-e8d5-432a-9430-0a270c02be3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:39.579369 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9903ba2-e8d5-432a-9430-0a270c02be3d None admin] RBAC: Adding query filter params (role.id=80400d1e5fdb44b3a92a468f6b62b248, scope.project.id=3a96e4f9e7134925af6e684f79748f71, user.id=f4e8b36563d1429d9d37523e65dafd7a) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:39.579676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9903ba2-e8d5-432a-9430-0a270c02be3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:39.581109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9903ba2-e8d5-432a-9430-0a270c02be3d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:39.599573 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 133/262] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:39 2022] GET /identity/v3/role_assignments?user.id=f4e8b36563d1429d9d37523e65dafd7a&scope.project.id=3a96e4f9e7134925af6e684f79748f71&role.id=80400d1e5fdb44b3a92a468f6b62b248 => generated 259 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:39.600219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9903ba2-e8d5-432a-9430-0a270c02be3d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:39.618674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff0d4e-2720-4a3f-af0b-0d321ecab3cb None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:39.702556 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa620d-ac73-4d2f-a604-c2cd7be50fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:39.703592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa620d-ac73-4d2f-a604-c2cd7be50fba None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:39.704091 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa620d-ac73-4d2f-a604-c2cd7be50fba None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:39.704576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa620d-ac73-4d2f-a604-c2cd7be50fba None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:39.705546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa620d-ac73-4d2f-a604-c2cd7be50fba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:39.706095 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa620d-ac73-4d2f-a604-c2cd7be50fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:39.707800 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa620d-ac73-4d2f-a604-c2cd7be50fba None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:39.736641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 130/263] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:52:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:39.742091 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa620d-ac73-4d2f-a604-c2cd7be50fba None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:39.791340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:39.796294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9903ba2-e8d5-432a-9430-0a270c02be3d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:39.888247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1a548-2a9e-4659-931d-3994c7ddd3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:39.888830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1a548-2a9e-4659-931d-3994c7ddd3e1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:39.889110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1a548-2a9e-4659-931d-3994c7ddd3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:39.889385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1a548-2a9e-4659-931d-3994c7ddd3e1 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:39.890602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e1a548-2a9e-4659-931d-3994c7ddd3e1 None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:39.892642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e1a548-2a9e-4659-931d-3994c7ddd3e1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:39.894609 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 134/264] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:39 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:39.901232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1a548-2a9e-4659-931d-3994c7ddd3e1 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:39.946885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa620d-ac73-4d2f-a604-c2cd7be50fba None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:39.988080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:39.996164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391821d-0b2f-492a-97d8-9898eb5c79b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:39.996801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391821d-0b2f-492a-97d8-9898eb5c79b1 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:39.997316 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391821d-0b2f-492a-97d8-9898eb5c79b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:39.997585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391821d-0b2f-492a-97d8-9898eb5c79b1 None admin] PATH_INFO: `/v3/users/ef0866e48d07470db813ba0449c751f9` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:39.998990 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391821d-0b2f-492a-97d8-9898eb5c79b1 None admin] RBAC: Authorizing `identity:get_user(user_id=ef0866e48d07470db813ba0449c751f9)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:40.000602 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391821d-0b2f-492a-97d8-9898eb5c79b1 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:40.006806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 131/265] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:39 2022] GET /identity/v3/users/ef0866e48d07470db813ba0449c751f9 => generated 283 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:40.016117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391821d-0b2f-492a-97d8-9898eb5c79b1 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:40.120664 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1a548-2a9e-4659-931d-3994c7ddd3e1 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:40.180046 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391821d-0b2f-492a-97d8-9898eb5c79b1 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:40.187520 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec65eaf-89eb-4355-bc36-5301d2d5ef88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:40.188158 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec65eaf-89eb-4355-bc36-5301d2d5ef88 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:40.188440 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec65eaf-89eb-4355-bc36-5301d2d5ef88 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:40.188711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec65eaf-89eb-4355-bc36-5301d2d5ef88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:40.189449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec65eaf-89eb-4355-bc36-5301d2d5ef88 None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=3a96e4f9e7134925af6e684f79748f71, user.id=ef0866e48d07470db813ba0449c751f9) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:40.189758 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec65eaf-89eb-4355-bc36-5301d2d5ef88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:40.191342 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec65eaf-89eb-4355-bc36-5301d2d5ef88 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:40.202454 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 135/266] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:39 2022] GET /identity/v3/role_assignments?user.id=ef0866e48d07470db813ba0449c751f9&scope.project.id=3a96e4f9e7134925af6e684f79748f71&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:40.234063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818e98e-fe54-4c42-816d-ce0196089cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:40.235032 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818e98e-fe54-4c42-816d-ce0196089cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:40.235503 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818e98e-fe54-4c42-816d-ce0196089cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:40.235950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818e98e-fe54-4c42-816d-ce0196089cb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:40.326284 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:40.332670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818e98e-fe54-4c42-816d-ce0196089cb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:40.334276 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818e98e-fe54-4c42-816d-ce0196089cb3 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:40.471581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 132/267] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:40.476823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec65eaf-89eb-4355-bc36-5301d2d5ef88 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:40.556826 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:40.642913 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec65eaf-89eb-4355-bc36-5301d2d5ef88 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:40.685350 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33e0e0-a56c-40f0-bf43-e44421ef072b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:40.686000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33e0e0-a56c-40f0-bf43-e44421ef072b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:40.686313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33e0e0-a56c-40f0-bf43-e44421ef072b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:40.686587 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33e0e0-a56c-40f0-bf43-e44421ef072b None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:40.687737 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33e0e0-a56c-40f0-bf43-e44421ef072b None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:40.689171 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33e0e0-a56c-40f0-bf43-e44421ef072b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:40.691171 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 136/268] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:40 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:40.693392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1299e0-ac1f-48f7-b57d-b8d485dac042 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:40.693734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1299e0-ac1f-48f7-b57d-b8d485dac042 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:40.694079 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1299e0-ac1f-48f7-b57d-b8d485dac042 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:40.695829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 133/269] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:40.697276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33e0e0-a56c-40f0-bf43-e44421ef072b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:40.703970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55760f8-ff91-4d82-9930-928a984af47e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:40.704325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55760f8-ff91-4d82-9930-928a984af47e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:40.704668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55760f8-ff91-4d82-9930-928a984af47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:40.744722 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a55760f8-ff91-4d82-9930-928a984af47e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:40.746999 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a55760f8-ff91-4d82-9930-928a984af47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:40.863088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:40.866543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33e0e0-a56c-40f0-bf43-e44421ef072b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:40.871291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 134/270] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:40.872872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f405f6-fd03-4b9a-8a58-b5a6956c84d3 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:40.873304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f405f6-fd03-4b9a-8a58-b5a6956c84d3 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:40.873723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f405f6-fd03-4b9a-8a58-b5a6956c84d3 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:40.877519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 135/271] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:40.881493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08c2e8-140f-43d7-8766-8608c075125f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:40.882616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08c2e8-140f-43d7-8766-8608c075125f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:40.883075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08c2e8-140f-43d7-8766-8608c075125f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:40.908224 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-ea08c2e8-140f-43d7-8766-8608c075125f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:40.909439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ea08c2e8-140f-43d7-8766-8608c075125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:40.919043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b88a8c-7593-4e3a-bd66-3ae979524798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:40.920424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b88a8c-7593-4e3a-bd66-3ae979524798 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:40.920707 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b88a8c-7593-4e3a-bd66-3ae979524798 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:40.920980 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b88a8c-7593-4e3a-bd66-3ae979524798 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:40.921721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b88a8c-7593-4e3a-bd66-3ae979524798 None admin] RBAC: Adding query filter params (role.id=d43148a29472443ba7dfe2424ec60baf, scope.project.id=3a96e4f9e7134925af6e684f79748f71, user.id=ef0866e48d07470db813ba0449c751f9) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:40.922030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b88a8c-7593-4e3a-bd66-3ae979524798 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:40.923726 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b88a8c-7593-4e3a-bd66-3ae979524798 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:40.931421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 137/272] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:40 2022] GET /identity/v3/role_assignments?user.id=ef0866e48d07470db813ba0449c751f9&scope.project.id=3a96e4f9e7134925af6e684f79748f71&role.id=d43148a29472443ba7dfe2424ec60baf => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:40.938479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:40.990997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 136/273] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:41.277081 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b88a8c-7593-4e3a-bd66-3ae979524798 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:41.322339 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08c2e8-140f-43d7-8766-8608c075125f None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:41.397074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b88a8c-7593-4e3a-bd66-3ae979524798 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:41.436469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1c8c8-b2cf-4917-9916-24829bf20bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:41.437557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1c8c8-b2cf-4917-9916-24829bf20bbc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:41.437840 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1c8c8-b2cf-4917-9916-24829bf20bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:41.438107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1c8c8-b2cf-4917-9916-24829bf20bbc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:41.449929 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1c8c8-b2cf-4917-9916-24829bf20bbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-92e1c8c8-b2cf-4917-9916-24829bf20bbc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:41.456101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:41.457764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:41.459216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:41.459216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:41.459216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:41.459216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:41.459216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:41.459216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:41.459216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:41.459216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:41.459216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:41.459216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:41.459216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:41.459216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:41.459216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 138/274] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:41.463620 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1c8c8-b2cf-4917-9916-24829bf20bbc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:41.492924 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:41.503221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08c2e8-140f-43d7-8766-8608c075125f None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:41.575814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d804f-93d6-4221-b970-52bb7897772d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:41.577427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d804f-93d6-4221-b970-52bb7897772d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:41.578217 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d804f-93d6-4221-b970-52bb7897772d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:41.579100 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d804f-93d6-4221-b970-52bb7897772d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:41.581363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d804f-93d6-4221-b970-52bb7897772d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:41.582572 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1c8c8-b2cf-4917-9916-24829bf20bbc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:41.584926 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d804f-93d6-4221-b970-52bb7897772d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:41.588622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 137/275] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:52:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:41.595134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d804f-93d6-4221-b970-52bb7897772d None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:41.631859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f186b-7e6e-40cb-9119-1e6988487298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:41.632635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f186b-7e6e-40cb-9119-1e6988487298 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:41.633016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f186b-7e6e-40cb-9119-1e6988487298 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:41.633466 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f186b-7e6e-40cb-9119-1e6988487298 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:41.635400 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:41.635560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f186b-7e6e-40cb-9119-1e6988487298 None admin] RBAC: Adding query filter params (name=member) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:41.636350 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f186b-7e6e-40cb-9119-1e6988487298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:41.638282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f186b-7e6e-40cb-9119-1e6988487298 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:41.655073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 139/276] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:41.657559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f186b-7e6e-40cb-9119-1e6988487298 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:41.737386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d804f-93d6-4221-b970-52bb7897772d None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:41.757484 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:41.776533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f186b-7e6e-40cb-9119-1e6988487298 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:41.789014 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f6dc4-22e9-4a33-b9c2-782b50c27a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:41.789747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f6dc4-22e9-4a33-b9c2-782b50c27a54 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:41.790026 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f6dc4-22e9-4a33-b9c2-782b50c27a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:41.790327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f6dc4-22e9-4a33-b9c2-782b50c27a54 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:41.790978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f6dc4-22e9-4a33-b9c2-782b50c27a54 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:41.793142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f6dc4-22e9-4a33-b9c2-782b50c27a54 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:41.817418 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10bd41-7f5a-4612-9d7a-66cd24f77d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:41.818513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10bd41-7f5a-4612-9d7a-66cd24f77d97 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:41.818989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10bd41-7f5a-4612-9d7a-66cd24f77d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:41.819454 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10bd41-7f5a-4612-9d7a-66cd24f77d97 None admin] PATH_INFO: `/v3/users/0da6c88bcdaa465dbf10d7c77465239c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:41.828379 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 138/277] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:52:41 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:52:41.841247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10bd41-7f5a-4612-9d7a-66cd24f77d97 None admin] RBAC: Authorizing `identity:get_user(user_id=0da6c88bcdaa465dbf10d7c77465239c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:41.842996 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10bd41-7f5a-4612-9d7a-66cd24f77d97 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:41.844828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 140/278] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:41 2022] GET /identity/v3/users/0da6c88bcdaa465dbf10d7c77465239c => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:41.851252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f6dc4-22e9-4a33-b9c2-782b50c27a54 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:41.959638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:41.981049 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f6dc4-22e9-4a33-b9c2-782b50c27a54 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:42.025113 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cae16f-87b7-4afe-90ca-0d3319983304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:42.025718 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cae16f-87b7-4afe-90ca-0d3319983304 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:42.025945 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cae16f-87b7-4afe-90ca-0d3319983304 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:42.026210 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cae16f-87b7-4afe-90ca-0d3319983304 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:42.026950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cae16f-87b7-4afe-90ca-0d3319983304 None admin] RBAC: Adding query filter params (role.id=e385f7d11fac4932a910ec0ef95e2a43, user.id=0da6c88bcdaa465dbf10d7c77465239c, scope.system=all) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:42.027320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cae16f-87b7-4afe-90ca-0d3319983304 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:42.028801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cae16f-87b7-4afe-90ca-0d3319983304 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:42.034806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 139/279] 10.222.0.62 () {56 vars in 1308 bytes} [Fri Sep 9 11:52:41 2022] GET /identity/v3/role_assignments?user.id=0da6c88bcdaa465dbf10d7c77465239c&scope.system=all&role.id=e385f7d11fac4932a910ec0ef95e2a43 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:42.120569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4b2df-3f8c-4ee2-a721-8b3411003b43 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:42.121078 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4b2df-3f8c-4ee2-a721-8b3411003b43 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:42.121078 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4b2df-3f8c-4ee2-a721-8b3411003b43 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:42.122712 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 141/280] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:42.134175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5f25d-8ff9-47d4-9096-a70faf3a63ae None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:42.134175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5f25d-8ff9-47d4-9096-a70faf3a63ae None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:42.134175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5f25d-8ff9-47d4-9096-a70faf3a63ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:42.155411 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-04c5f25d-8ff9-47d4-9096-a70faf3a63ae None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:42.156862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-04c5f25d-8ff9-47d4-9096-a70faf3a63ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:42.239746 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 140/281] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:42.578215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4b2df-3f8c-4ee2-a721-8b3411003b43 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:42.640168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7d5c3-6187-4c25-8c97-7e781ffc4f2e None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:42.640512 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7d5c3-6187-4c25-8c97-7e781ffc4f2e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:42.640727 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7d5c3-6187-4c25-8c97-7e781ffc4f2e None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:42.642441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 141/282] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:42.650904 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cb9ec-d451-4dde-9743-384c7a44ed55 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:42.651179 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cb9ec-d451-4dde-9743-384c7a44ed55 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:42.651450 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cb9ec-d451-4dde-9743-384c7a44ed55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:42.673462 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-047cb9ec-d451-4dde-9743-384c7a44ed55 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:42.674702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-047cb9ec-d451-4dde-9743-384c7a44ed55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:42.740760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4b2df-3f8c-4ee2-a721-8b3411003b43 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:42.794847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 142/283] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:42.800162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef4bc2-097f-4f88-89ae-084fa749e494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:42.800806 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef4bc2-097f-4f88-89ae-084fa749e494 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:42.801086 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef4bc2-097f-4f88-89ae-084fa749e494 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:42.801362 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef4bc2-097f-4f88-89ae-084fa749e494 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:42.812457 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef4bc2-097f-4f88-89ae-084fa749e494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-0bef4bc2-097f-4f88-89ae-084fa749e494 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:42.817341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:42.818288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:42.819070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:42.819070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:42.819070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:42.819070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:42.819070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:42.819070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:42.819070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:42.819070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:42.819070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:42.819070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:42.819070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:42.819070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:42.819070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 142/284] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:42.824606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cb9ec-d451-4dde-9743-384c7a44ed55 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:42.838490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:42.984013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cb9ec-d451-4dde-9743-384c7a44ed55 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:43.035744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc6273-b557-421b-bd74-e792a97e1e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:43.037223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc6273-b557-421b-bd74-e792a97e1e06 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:43.037728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc6273-b557-421b-bd74-e792a97e1e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:43.038227 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc6273-b557-421b-bd74-e792a97e1e06 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:43.039099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc6273-b557-421b-bd74-e792a97e1e06 None admin] RBAC: Adding query filter params (name=member) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:43.040133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc6273-b557-421b-bd74-e792a97e1e06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:43.041855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc6273-b557-421b-bd74-e792a97e1e06 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:43.050784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 143/285] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:43.057365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef4bc2-097f-4f88-89ae-084fa749e494 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:43.064221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:43.101983 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc6273-b557-421b-bd74-e792a97e1e06 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:43.256259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef4bc2-097f-4f88-89ae-084fa749e494 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:43.261513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:43.318600 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956f5f5-8e59-40ee-aa83-7bb84d84db33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:43.320188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956f5f5-8e59-40ee-aa83-7bb84d84db33 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:43.320882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956f5f5-8e59-40ee-aa83-7bb84d84db33 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:43.322532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956f5f5-8e59-40ee-aa83-7bb84d84db33 None admin] PATH_INFO: `/v3/users/9cc48f17f5f246d8a9eeed99ff863740` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:43.327542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956f5f5-8e59-40ee-aa83-7bb84d84db33 None admin] RBAC: Authorizing `identity:get_user(user_id=9cc48f17f5f246d8a9eeed99ff863740)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:43.327542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956f5f5-8e59-40ee-aa83-7bb84d84db33 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:43.327542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 143/286] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:43 2022] GET /identity/v3/users/9cc48f17f5f246d8a9eeed99ff863740 => generated 297 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:43.330728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc6273-b557-421b-bd74-e792a97e1e06 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:43.332822 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956f5f5-8e59-40ee-aa83-7bb84d84db33 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:43.408596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:43.413353 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d34168-5193-46cb-962b-5940b18082cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:43.414005 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d34168-5193-46cb-962b-5940b18082cd None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:43.414312 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d34168-5193-46cb-962b-5940b18082cd None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:43.414607 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d34168-5193-46cb-962b-5940b18082cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:43.438096 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d34168-5193-46cb-962b-5940b18082cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:43.445107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956f5f5-8e59-40ee-aa83-7bb84d84db33 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-46d34168-5193-46cb-962b-5940b18082cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:43.451771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:43.452668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:43.453481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:43.453481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:43.453481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:43.453481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:43.453481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:43.453481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:43.453481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:43.453481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:43.453481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:43.453481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:43.453481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:43.453481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:43.453481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 144/287] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:43.459022 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d34168-5193-46cb-962b-5940b18082cd None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:43.488762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987799e1-17c5-4604-a7de-4c833fb24d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:43.489724 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987799e1-17c5-4604-a7de-4c833fb24d18 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:43.490428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987799e1-17c5-4604-a7de-4c833fb24d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:43.491571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987799e1-17c5-4604-a7de-4c833fb24d18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:43.615456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987799e1-17c5-4604-a7de-4c833fb24d18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:43.616928 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987799e1-17c5-4604-a7de-4c833fb24d18 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:43.665926 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d34168-5193-46cb-962b-5940b18082cd None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:43.725923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:43.736501 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcacbaf-3d23-4c10-b1b3-49b687c97333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:43.737112 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcacbaf-3d23-4c10-b1b3-49b687c97333 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:43.737384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcacbaf-3d23-4c10-b1b3-49b687c97333 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:43.737653 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcacbaf-3d23-4c10-b1b3-49b687c97333 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:43.738338 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcacbaf-3d23-4c10-b1b3-49b687c97333 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:43.738635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcacbaf-3d23-4c10-b1b3-49b687c97333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:43.740002 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcacbaf-3d23-4c10-b1b3-49b687c97333 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:43.751951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:43.761088 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 145/288] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:43.767331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcacbaf-3d23-4c10-b1b3-49b687c97333 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:43.804054 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 144/289] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:52:43.810108 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987799e1-17c5-4604-a7de-4c833fb24d18 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:43.955282 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcacbaf-3d23-4c10-b1b3-49b687c97333 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:43.981947 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987799e1-17c5-4604-a7de-4c833fb24d18 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:43.992937 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03155c50-961b-49b6-92ec-40942d428437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:43.993572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03155c50-961b-49b6-92ec-40942d428437 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:43.993850 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03155c50-961b-49b6-92ec-40942d428437 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:43.994144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03155c50-961b-49b6-92ec-40942d428437 None admin] PATH_INFO: `/v3/users/f4e8b36563d1429d9d37523e65dafd7a` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:43.995283 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03155c50-961b-49b6-92ec-40942d428437 None admin] RBAC: Authorizing `identity:get_user(user_id=f4e8b36563d1429d9d37523e65dafd7a)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:43.996881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03155c50-961b-49b6-92ec-40942d428437 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:43.998819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 146/290] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:43 2022] GET /identity/v3/users/f4e8b36563d1429d9d37523e65dafd7a => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:44.005513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03155c50-961b-49b6-92ec-40942d428437 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:44.050406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c82af-dce0-4099-86ce-5865213b1e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:44.051045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c82af-dce0-4099-86ce-5865213b1e99 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:44.051315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c82af-dce0-4099-86ce-5865213b1e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:44.051584 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c82af-dce0-4099-86ce-5865213b1e99 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:44.054813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c82af-dce0-4099-86ce-5865213b1e99 None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:44.056281 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c82af-dce0-4099-86ce-5865213b1e99 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:44.062025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 145/291] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:43 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:44.067892 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c82af-dce0-4099-86ce-5865213b1e99 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:44.131589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:44.142107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:44.239021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03155c50-961b-49b6-92ec-40942d428437 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:44.254581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c82af-dce0-4099-86ce-5865213b1e99 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:44.313256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c3abc-e2b4-4092-9ab1-28fd86331170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:44.314297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c3abc-e2b4-4092-9ab1-28fd86331170 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:44.314779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c3abc-e2b4-4092-9ab1-28fd86331170 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:44.315271 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c3abc-e2b4-4092-9ab1-28fd86331170 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:44.317186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c3abc-e2b4-4092-9ab1-28fd86331170 None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:44.319366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c3abc-e2b4-4092-9ab1-28fd86331170 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:44.323502 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 147/292] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:44 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:44.324397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1a2fe-e2fe-4cdc-aab6-1ac881b513fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:44.325655 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1a2fe-e2fe-4cdc-aab6-1ac881b513fc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:44.326202 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1a2fe-e2fe-4cdc-aab6-1ac881b513fc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:44.326672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1a2fe-e2fe-4cdc-aab6-1ac881b513fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:44.327661 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1a2fe-e2fe-4cdc-aab6-1ac881b513fc None admin] RBAC: Adding query filter params (role.id=e385f7d11fac4932a910ec0ef95e2a43, scope.project.id=3a96e4f9e7134925af6e684f79748f71, user.id=9cc48f17f5f246d8a9eeed99ff863740) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:44.328687 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c3abc-e2b4-4092-9ab1-28fd86331170 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:44.329219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1a2fe-e2fe-4cdc-aab6-1ac881b513fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:44.330885 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1a2fe-e2fe-4cdc-aab6-1ac881b513fc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:44.346265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 146/293] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:44 2022] GET /identity/v3/role_assignments?user.id=9cc48f17f5f246d8a9eeed99ff863740&scope.project.id=3a96e4f9e7134925af6e684f79748f71&role.id=e385f7d11fac4932a910ec0ef95e2a43 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:44.488939 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c3abc-e2b4-4092-9ab1-28fd86331170 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:44.521996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:44.526397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706a355-1d6b-4018-96d5-e7501ba05d2b None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:44.526719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706a355-1d6b-4018-96d5-e7501ba05d2b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:44.526952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706a355-1d6b-4018-96d5-e7501ba05d2b None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:44.528452 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 147/294] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:52:44.534038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce61627-97ff-49a2-8461-35696b284861 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:44.534354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce61627-97ff-49a2-8461-35696b284861 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:44.534628 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce61627-97ff-49a2-8461-35696b284861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:44.534826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58101b0d-f44d-4494-a456-f80a1826e992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:44.535746 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58101b0d-f44d-4494-a456-f80a1826e992 None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:44.536296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58101b0d-f44d-4494-a456-f80a1826e992 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:44.536763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58101b0d-f44d-4494-a456-f80a1826e992 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71/users/f4e8b36563d1429d9d37523e65dafd7a/roles/80400d1e5fdb44b3a92a468f6b62b248` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:44.539107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58101b0d-f44d-4494-a456-f80a1826e992 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a96e4f9e7134925af6e684f79748f71, user_id=f4e8b36563d1429d9d37523e65dafd7a, role_id=80400d1e5fdb44b3a92a468f6b62b248)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:44.541766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58101b0d-f44d-4494-a456-f80a1826e992 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:44.554410 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-bce61627-97ff-49a2-8461-35696b284861 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:44.556226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bce61627-97ff-49a2-8461-35696b284861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:44.561017 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 148/295] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:52:44 2022] PUT /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71/users/f4e8b36563d1429d9d37523e65dafd7a/roles/80400d1e5fdb44b3a92a468f6b62b248 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:52:44.612202 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5d180-59da-484c-a2a2-f8d6c9ee4b1d None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:44.613163 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5d180-59da-484c-a2a2-f8d6c9ee4b1d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:44.613650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5d180-59da-484c-a2a2-f8d6c9ee4b1d None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:44.616129 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 149/296] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:44.622679 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6ccfd-bffc-445e-b2a2-1042e32e483f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:44.622679 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6ccfd-bffc-445e-b2a2-1042e32e483f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:44.622952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6ccfd-bffc-445e-b2a2-1042e32e483f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:44.643001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-eea6ccfd-bffc-445e-b2a2-1042e32e483f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:44.644212 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-eea6ccfd-bffc-445e-b2a2-1042e32e483f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:44.644890 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 148/297] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:44.739257 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 150/298] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:44.811960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfcdea5-832a-4bb0-b982-39e673b4201a None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:44.812563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfcdea5-832a-4bb0-b982-39e673b4201a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:44.812840 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfcdea5-832a-4bb0-b982-39e673b4201a None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:44.814504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 149/299] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:44.819804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42528dcb-067d-4192-b8f0-91ac437978c0 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:44.820089 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42528dcb-067d-4192-b8f0-91ac437978c0 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:44.820363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42528dcb-067d-4192-b8f0-91ac437978c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:44.840447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-42528dcb-067d-4192-b8f0-91ac437978c0 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:44.841728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-42528dcb-067d-4192-b8f0-91ac437978c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:44.921335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 151/300] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:44.977426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfcdea5-832a-4bb0-b982-39e673b4201a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:45.020672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:45.029971 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42528dcb-067d-4192-b8f0-91ac437978c0 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:45.094569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:45.108360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfcdea5-832a-4bb0-b982-39e673b4201a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:45.148307 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e443f-7504-42b1-9d97-89b414cdb086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:45.149842 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e443f-7504-42b1-9d97-89b414cdb086 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:45.150330 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e443f-7504-42b1-9d97-89b414cdb086 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:45.150821 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e443f-7504-42b1-9d97-89b414cdb086 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:45.161297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42528dcb-067d-4192-b8f0-91ac437978c0 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:45.165513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e443f-7504-42b1-9d97-89b414cdb086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-660e443f-7504-42b1-9d97-89b414cdb086 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:45.171211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:45.172284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:45.173248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:45.173248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:45.173248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:45.173248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:45.173248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:45.173248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:45.173248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:45.173248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:45.173248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:45.173248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:45.173248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:45.173248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:45.173248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 150/301] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:45.179650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e443f-7504-42b1-9d97-89b414cdb086 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:45.202085 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42643b45-632b-438e-aef5-5a428139feb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:45.202831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42643b45-632b-438e-aef5-5a428139feb7 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:45.203115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42643b45-632b-438e-aef5-5a428139feb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:45.203389 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42643b45-632b-438e-aef5-5a428139feb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:45.217548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42643b45-632b-438e-aef5-5a428139feb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-42643b45-632b-438e-aef5-5a428139feb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:45.222840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:45.223713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:45.224497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:45.224497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:45.224497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:45.224497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:45.224497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:45.224497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:45.224497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:45.224497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:45.224497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:45.224497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:45.224497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:45.224497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:45.224497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 152/302] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:45.225130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42643b45-632b-438e-aef5-5a428139feb7 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:45.293776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:45.301397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e443f-7504-42b1-9d97-89b414cdb086 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:45.339267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230900c-ca09-41f4-a54e-6d3fea76ec54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:45.339923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42643b45-632b-438e-aef5-5a428139feb7 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:45.340021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230900c-ca09-41f4-a54e-6d3fea76ec54 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:45.340208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230900c-ca09-41f4-a54e-6d3fea76ec54 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:45.340392 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230900c-ca09-41f4-a54e-6d3fea76ec54 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:45.341053 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230900c-ca09-41f4-a54e-6d3fea76ec54 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:45.341354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230900c-ca09-41f4-a54e-6d3fea76ec54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:45.342765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230900c-ca09-41f4-a54e-6d3fea76ec54 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:45.350723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:45.352283 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 151/303] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:45.352624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230900c-ca09-41f4-a54e-6d3fea76ec54 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:45.382161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf836b-013d-43ff-8b7d-91b0dd220a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:45.382713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf836b-013d-43ff-8b7d-91b0dd220a82 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:45.382997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf836b-013d-43ff-8b7d-91b0dd220a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:45.383285 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf836b-013d-43ff-8b7d-91b0dd220a82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:45.384875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf836b-013d-43ff-8b7d-91b0dd220a82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:45.386329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf836b-013d-43ff-8b7d-91b0dd220a82 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:45.387892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 153/304] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:52:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:45.393182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf836b-013d-43ff-8b7d-91b0dd220a82 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:45.461819 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230900c-ca09-41f4-a54e-6d3fea76ec54 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:45.499796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf836b-013d-43ff-8b7d-91b0dd220a82 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:45.500766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec65884-5daa-49b1-83d3-6367a0d68331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:45.501688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec65884-5daa-49b1-83d3-6367a0d68331 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:45.502226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec65884-5daa-49b1-83d3-6367a0d68331 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:45.502592 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec65884-5daa-49b1-83d3-6367a0d68331 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:45.503284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec65884-5daa-49b1-83d3-6367a0d68331 None admin] RBAC: Adding query filter params (name=member) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:45.503591 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec65884-5daa-49b1-83d3-6367a0d68331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:45.505006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec65884-5daa-49b1-83d3-6367a0d68331 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:45.520526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 152/305] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:45.520526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec65884-5daa-49b1-83d3-6367a0d68331 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:45.536528 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858277b6-7253-4b33-b00e-d4376cf31e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:45.537168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858277b6-7253-4b33-b00e-d4376cf31e00 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:45.537446 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858277b6-7253-4b33-b00e-d4376cf31e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:45.537720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858277b6-7253-4b33-b00e-d4376cf31e00 None admin] PATH_INFO: `/v3/users/9731d2fb17bf425c99b22f165294d113` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:45.542397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:45.555489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858277b6-7253-4b33-b00e-d4376cf31e00 None admin] RBAC: Authorizing `identity:get_user(user_id=9731d2fb17bf425c99b22f165294d113)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:45.557036 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858277b6-7253-4b33-b00e-d4376cf31e00 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:45.558812 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 154/306] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:45 2022] GET /identity/v3/users/9731d2fb17bf425c99b22f165294d113 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:45.559619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858277b6-7253-4b33-b00e-d4376cf31e00 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:45.598694 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:45.649989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec65884-5daa-49b1-83d3-6367a0d68331 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:45.673602 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858277b6-7253-4b33-b00e-d4376cf31e00 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:45.685107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b93c9e-db28-42b3-bc77-7964a7c5f7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:45.685727 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b93c9e-db28-42b3-bc77-7964a7c5f7d0 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:45.686006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b93c9e-db28-42b3-bc77-7964a7c5f7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:45.686313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b93c9e-db28-42b3-bc77-7964a7c5f7d0 None admin] PATH_INFO: `/v3/groups` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:45.686949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b93c9e-db28-42b3-bc77-7964a7c5f7d0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:45.688383 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b93c9e-db28-42b3-bc77-7964a7c5f7d0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:45.705320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 153/307] 10.222.0.62 () {60 vars in 1134 bytes} [Fri Sep 9 11:52:45 2022] POST /identity/v3/groups => generated 229 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:52:45.707109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b93c9e-db28-42b3-bc77-7964a7c5f7d0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:45.725851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b26d3-7d2f-4a49-9865-52416cd92554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:45.727032 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b26d3-7d2f-4a49-9865-52416cd92554 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:45.727590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b26d3-7d2f-4a49-9865-52416cd92554 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:45.728066 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b26d3-7d2f-4a49-9865-52416cd92554 None admin] PATH_INFO: `/v3/users/0da6c88bcdaa465dbf10d7c77465239c` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:45.729814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b26d3-7d2f-4a49-9865-52416cd92554 None admin] RBAC: Authorizing `identity:get_user(user_id=0da6c88bcdaa465dbf10d7c77465239c)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:45.731642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b26d3-7d2f-4a49-9865-52416cd92554 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:45.734089 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 155/308] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:45 2022] GET /identity/v3/users/0da6c88bcdaa465dbf10d7c77465239c => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:45.740233 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b26d3-7d2f-4a49-9865-52416cd92554 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:45.821443 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:45.860613 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b93c9e-db28-42b3-bc77-7964a7c5f7d0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:45.877892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b26d3-7d2f-4a49-9865-52416cd92554 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:45.880243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:45.926321 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a0260-622a-4e4f-bf22-e3c2af0f6e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:45.926985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a0260-622a-4e4f-bf22-e3c2af0f6e73 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:45.927248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98117b7c-7825-455e-8703-b09da150c7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:45.927389 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a0260-622a-4e4f-bf22-e3c2af0f6e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:45.927600 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a0260-622a-4e4f-bf22-e3c2af0f6e73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:45.928262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a0260-622a-4e4f-bf22-e3c2af0f6e73 None admin] RBAC: Adding query filter params (role.id=80400d1e5fdb44b3a92a468f6b62b248, user.id=9731d2fb17bf425c99b22f165294d113, scope.system=all) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:45.928570 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a0260-622a-4e4f-bf22-e3c2af0f6e73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:45.928773 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98117b7c-7825-455e-8703-b09da150c7e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:45.929055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98117b7c-7825-455e-8703-b09da150c7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:45.929332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98117b7c-7825-455e-8703-b09da150c7e6 None admin] PATH_INFO: `/v3/system/users/0da6c88bcdaa465dbf10d7c77465239c/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:45.930010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a0260-622a-4e4f-bf22-e3c2af0f6e73 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:45.930884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98117b7c-7825-455e-8703-b09da150c7e6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0da6c88bcdaa465dbf10d7c77465239c, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:45.932478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98117b7c-7825-455e-8703-b09da150c7e6 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:45.938954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 154/309] 10.222.0.62 () {56 vars in 1308 bytes} [Fri Sep 9 11:52:45 2022] GET /identity/v3/role_assignments?user.id=9731d2fb17bf425c99b22f165294d113&scope.system=all&role.id=80400d1e5fdb44b3a92a468f6b62b248 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:45.947825 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 156/310] 10.222.0.62 () {58 vars in 1334 bytes} [Fri Sep 9 11:52:45 2022] PUT /identity/v3/system/users/0da6c88bcdaa465dbf10d7c77465239c/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:52:46.889370 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd7a76-6e39-4661-800f-8cc511067e9e None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:46.890273 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd7a76-6e39-4661-800f-8cc511067e9e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:46.890723 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd7a76-6e39-4661-800f-8cc511067e9e None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:46.892778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 155/311] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:46.898544 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6634eef-9bf9-4717-94eb-24a766e62877 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:46.899878 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6634eef-9bf9-4717-94eb-24a766e62877 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:46.900373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6634eef-9bf9-4717-94eb-24a766e62877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:46.920758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a6634eef-9bf9-4717-94eb-24a766e62877 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:46.922365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a6634eef-9bf9-4717-94eb-24a766e62877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:46.995066 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 157/312] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:47.271312 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd7a76-6e39-4661-800f-8cc511067e9e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:47.379971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd7a76-6e39-4661-800f-8cc511067e9e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:47.415722 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3c77e-3151-49f8-9cfa-9c1e30e664ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:47.416369 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3c77e-3151-49f8-9cfa-9c1e30e664ad None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:47.416650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3c77e-3151-49f8-9cfa-9c1e30e664ad None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:47.416917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3c77e-3151-49f8-9cfa-9c1e30e664ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:47.427351 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3c77e-3151-49f8-9cfa-9c1e30e664ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-77e3c77e-3151-49f8-9cfa-9c1e30e664ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:47.432293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:47.433194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:47.434006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:47.434006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:47.434006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:47.434006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:47.434006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:47.434006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:47.434006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:47.434006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:47.434006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:47.434006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:47.434006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:47.434006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:47.434006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 156/313] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:47.440963 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6634eef-9bf9-4717-94eb-24a766e62877 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:47.538584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:47.545030 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6634eef-9bf9-4717-94eb-24a766e62877 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:47.582024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba010d6e-b183-4b33-b13c-9b0556b44633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:47.582667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba010d6e-b183-4b33-b13c-9b0556b44633 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:47.582944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba010d6e-b183-4b33-b13c-9b0556b44633 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:47.583212 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba010d6e-b183-4b33-b13c-9b0556b44633 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:47.583877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba010d6e-b183-4b33-b13c-9b0556b44633 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:47.584175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba010d6e-b183-4b33-b13c-9b0556b44633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:47.585541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba010d6e-b183-4b33-b13c-9b0556b44633 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:47.594510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 158/314] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:47.600032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3c77e-3151-49f8-9cfa-9c1e30e664ad None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:47.604603 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:47.708760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3c77e-3151-49f8-9cfa-9c1e30e664ad None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:47.745578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52b544-853c-434e-a8af-a61c3bcc5b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:47.746618 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52b544-853c-434e-a8af-a61c3bcc5b22 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:47.747102 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52b544-853c-434e-a8af-a61c3bcc5b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:47.747591 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52b544-853c-434e-a8af-a61c3bcc5b22 None admin] PATH_INFO: `/v3/users/f4e8b36563d1429d9d37523e65dafd7a` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:47.748926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52b544-853c-434e-a8af-a61c3bcc5b22 None admin] RBAC: Authorizing `identity:get_user(user_id=f4e8b36563d1429d9d37523e65dafd7a)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:47.750596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52b544-853c-434e-a8af-a61c3bcc5b22 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:47.752831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 157/315] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:47 2022] GET /identity/v3/users/f4e8b36563d1429d9d37523e65dafd7a => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:47.758530 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba010d6e-b183-4b33-b13c-9b0556b44633 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:47.863550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba010d6e-b183-4b33-b13c-9b0556b44633 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:47.899730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd11e9-b9bb-4d4b-a5e6-cef7896f2cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:47.900362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd11e9-b9bb-4d4b-a5e6-cef7896f2cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:47.900606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd11e9-b9bb-4d4b-a5e6-cef7896f2cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:47.900877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd11e9-b9bb-4d4b-a5e6-cef7896f2cd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:47.931020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:47.972165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd11e9-b9bb-4d4b-a5e6-cef7896f2cd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:47.973641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd11e9-b9bb-4d4b-a5e6-cef7896f2cd7 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:48.087277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 159/316] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:48.093294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52b544-853c-434e-a8af-a61c3bcc5b22 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:48.170891 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:48.199585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe0e57-f371-45c7-a226-dcbf6b94b99f None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:48.199887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe0e57-f371-45c7-a226-dcbf6b94b99f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:48.200327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe0e57-f371-45c7-a226-dcbf6b94b99f None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:48.201967 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 160/317] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:48.207352 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e02c07-2454-4f98-91af-6ed04f449997 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:48.207636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e02c07-2454-4f98-91af-6ed04f449997 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:48.207910 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e02c07-2454-4f98-91af-6ed04f449997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:48.209029 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52b544-853c-434e-a8af-a61c3bcc5b22 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:48.228017 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-80e02c07-2454-4f98-91af-6ed04f449997 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:48.229509 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-80e02c07-2454-4f98-91af-6ed04f449997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:48.249054 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be14b814-a8f2-4bf5-bf30-eb1b263276c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:48.249687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14b814-a8f2-4bf5-bf30-eb1b263276c7 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:48.249959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14b814-a8f2-4bf5-bf30-eb1b263276c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:48.250253 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14b814-a8f2-4bf5-bf30-eb1b263276c7 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:48.251366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be14b814-a8f2-4bf5-bf30-eb1b263276c7 None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:48.252779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be14b814-a8f2-4bf5-bf30-eb1b263276c7 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:48.254430 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 158/318] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:48 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:48.255534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb7e1b-64b4-4d78-9f45-90f34be5aa4c None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:48.255806 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb7e1b-64b4-4d78-9f45-90f34be5aa4c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:48.256069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb7e1b-64b4-4d78-9f45-90f34be5aa4c None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:48.257398 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 159/319] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:48.258795 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7c45a-a011-498a-a3a2-b8c031d09893 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:48.259071 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7c45a-a011-498a-a3a2-b8c031d09893 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:48.259332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7c45a-a011-498a-a3a2-b8c031d09893 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:48.260514 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 160/320] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:48.263015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7c45a-a011-498a-a3a2-b8c031d09893 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:48.294214 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:48.308213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 161/321] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:48.309433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3498d8-ae3f-4eec-9088-1f4b4284722a None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:48.309711 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3498d8-ae3f-4eec-9088-1f4b4284722a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:48.309978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3498d8-ae3f-4eec-9088-1f4b4284722a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:48.328030 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-db3498d8-ae3f-4eec-9088-1f4b4284722a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:48.329201 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-db3498d8-ae3f-4eec-9088-1f4b4284722a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:48.371264 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7c45a-a011-498a-a3a2-b8c031d09893 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:48.402941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 162/322] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:48.404094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f2e0b-2786-4b21-b026-c4a4ca86a7d1 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:48.404375 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f2e0b-2786-4b21-b026-c4a4ca86a7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:48.404642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f2e0b-2786-4b21-b026-c4a4ca86a7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:48.408263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0575e7-562e-407a-bbe2-e9787ac13e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:48.408870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0575e7-562e-407a-bbe2-e9787ac13e9d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:48.409148 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0575e7-562e-407a-bbe2-e9787ac13e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:48.409418 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0575e7-562e-407a-bbe2-e9787ac13e9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:48.414428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0575e7-562e-407a-bbe2-e9787ac13e9d None admin] RBAC: Adding query filter params (role.id=80400d1e5fdb44b3a92a468f6b62b248, scope.project.id=3a96e4f9e7134925af6e684f79748f71, user.id=f4e8b36563d1429d9d37523e65dafd7a) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:48.414428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0575e7-562e-407a-bbe2-e9787ac13e9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:48.414428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0575e7-562e-407a-bbe2-e9787ac13e9d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:48.424026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 161/323] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:52:48 2022] GET /identity/v3/role_assignments?user.id=f4e8b36563d1429d9d37523e65dafd7a&scope.project.id=3a96e4f9e7134925af6e684f79748f71&role.id=80400d1e5fdb44b3a92a468f6b62b248 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:48.433565 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-457f2e0b-2786-4b21-b026-c4a4ca86a7d1 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:48.435331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-457f2e0b-2786-4b21-b026-c4a4ca86a7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:48.519704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 163/324] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:48.585918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0575e7-562e-407a-bbe2-e9787ac13e9d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:48.589905 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:48.676159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f2e0b-2786-4b21-b026-c4a4ca86a7d1 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:48.706108 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0575e7-562e-407a-bbe2-e9787ac13e9d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:48.741160 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:48.750015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30520f-327c-48b5-80e5-63aa8a21df5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:48.750680 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30520f-327c-48b5-80e5-63aa8a21df5b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:48.750957 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30520f-327c-48b5-80e5-63aa8a21df5b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:48.751225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30520f-327c-48b5-80e5-63aa8a21df5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:48.752323 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30520f-327c-48b5-80e5-63aa8a21df5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:48.753756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30520f-327c-48b5-80e5-63aa8a21df5b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:48.755357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 162/325] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:52:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:48.762035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30520f-327c-48b5-80e5-63aa8a21df5b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:48.800392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f2e0b-2786-4b21-b026-c4a4ca86a7d1 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:48.833943 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9d798-eb8a-43fa-97b0-f871f6355a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:48.834476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:48.834613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9d798-eb8a-43fa-97b0-f871f6355a27 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:48.834868 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9d798-eb8a-43fa-97b0-f871f6355a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:48.835139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9d798-eb8a-43fa-97b0-f871f6355a27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:48.845877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9d798-eb8a-43fa-97b0-f871f6355a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-e0d9d798-eb8a-43fa-97b0-f871f6355a27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:48.850798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:48.851667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:48.852735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:48.852735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:48.852735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:48.852735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:48.852735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:48.852735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:48.852735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:48.852735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:48.852735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:48.852735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:48.852735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:48.852735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:48.852735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 164/326] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:52:48.852735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9d798-eb8a-43fa-97b0-f871f6355a27 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:48.869017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30520f-327c-48b5-80e5-63aa8a21df5b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:48.903280 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0fbfa4-71db-4f35-9ab0-c79934af9991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:48.903891 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0fbfa4-71db-4f35-9ab0-c79934af9991 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:48.904171 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0fbfa4-71db-4f35-9ab0-c79934af9991 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:48.904447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0fbfa4-71db-4f35-9ab0-c79934af9991 None admin] PATH_INFO: `/v3/groups` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:48.905076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0fbfa4-71db-4f35-9ab0-c79934af9991 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:48.906542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0fbfa4-71db-4f35-9ab0-c79934af9991 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:48.934229 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 163/327] 10.222.0.62 () {60 vars in 1134 bytes} [Fri Sep 9 11:52:48 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:52:48.934229 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0fbfa4-71db-4f35-9ab0-c79934af9991 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:48.987105 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9d798-eb8a-43fa-97b0-f871f6355a27 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:48.999118 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:49.028239 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17552df-526c-4ab1-bc2f-0acb8c405481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:49.028959 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17552df-526c-4ab1-bc2f-0acb8c405481 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:49.029236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17552df-526c-4ab1-bc2f-0acb8c405481 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:49.029559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17552df-526c-4ab1-bc2f-0acb8c405481 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:49.039057 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0fbfa4-71db-4f35-9ab0-c79934af9991 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:49.041429 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17552df-526c-4ab1-bc2f-0acb8c405481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-d17552df-526c-4ab1-bc2f-0acb8c405481 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:49.046510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:49.047494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:49.048287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:49.048287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:49.048287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:49.048287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:49.048287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:49.048287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:49.048287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:49.048287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:49.048287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:49.048287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:49.048287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:49.048287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:49.048287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 165/328] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:49.053822 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17552df-526c-4ab1-bc2f-0acb8c405481 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:49.076814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:49.088058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335d24f-3a2f-4ffb-9acd-ca7577e1f458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:49.088687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335d24f-3a2f-4ffb-9acd-ca7577e1f458 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:49.088968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335d24f-3a2f-4ffb-9acd-ca7577e1f458 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:49.089241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335d24f-3a2f-4ffb-9acd-ca7577e1f458 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:49.089905 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335d24f-3a2f-4ffb-9acd-ca7577e1f458 None admin] RBAC: Adding query filter params (name=member) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:49.090235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335d24f-3a2f-4ffb-9acd-ca7577e1f458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:49.092167 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335d24f-3a2f-4ffb-9acd-ca7577e1f458 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:49.102911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 164/329] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:49.108982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335d24f-3a2f-4ffb-9acd-ca7577e1f458 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:49.175389 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17552df-526c-4ab1-bc2f-0acb8c405481 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:49.223807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40c371-99f0-47eb-80ec-10fb10f3d0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:49.224571 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40c371-99f0-47eb-80ec-10fb10f3d0cf None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:49.224873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40c371-99f0-47eb-80ec-10fb10f3d0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:49.225126 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40c371-99f0-47eb-80ec-10fb10f3d0cf None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:49.225904 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40c371-99f0-47eb-80ec-10fb10f3d0cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:49.228164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40c371-99f0-47eb-80ec-10fb10f3d0cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:49.229159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40c371-99f0-47eb-80ec-10fb10f3d0cf None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:49.239460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335d24f-3a2f-4ffb-9acd-ca7577e1f458 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:49.240180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 166/330] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:49.245798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40c371-99f0-47eb-80ec-10fb10f3d0cf None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:49.286348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:49.290303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12da8c-473a-42c8-ba69-5e0b832fbb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:49.290955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12da8c-473a-42c8-ba69-5e0b832fbb25 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:49.291234 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12da8c-473a-42c8-ba69-5e0b832fbb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:49.291505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12da8c-473a-42c8-ba69-5e0b832fbb25 None admin] PATH_INFO: `/v3/users/0da6c88bcdaa465dbf10d7c77465239c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:49.292797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12da8c-473a-42c8-ba69-5e0b832fbb25 None admin] RBAC: Authorizing `identity:get_user(user_id=0da6c88bcdaa465dbf10d7c77465239c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:49.294387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12da8c-473a-42c8-ba69-5e0b832fbb25 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:49.296345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 165/331] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:49 2022] GET /identity/v3/users/0da6c88bcdaa465dbf10d7c77465239c => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:49.303106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12da8c-473a-42c8-ba69-5e0b832fbb25 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:49.384641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40c371-99f0-47eb-80ec-10fb10f3d0cf None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:49.393168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:49.432519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693a4a8-f7e5-4e37-ac0a-8d023ee8d519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:49.433285 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693a4a8-f7e5-4e37-ac0a-8d023ee8d519 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:49.433561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693a4a8-f7e5-4e37-ac0a-8d023ee8d519 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:49.433838 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693a4a8-f7e5-4e37-ac0a-8d023ee8d519 None admin] PATH_INFO: `/v3/users/9731d2fb17bf425c99b22f165294d113` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:49.435035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3693a4a8-f7e5-4e37-ac0a-8d023ee8d519 None admin] RBAC: Authorizing `identity:get_user(user_id=9731d2fb17bf425c99b22f165294d113)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:49.436593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3693a4a8-f7e5-4e37-ac0a-8d023ee8d519 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:49.438422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 167/332] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:49 2022] GET /identity/v3/users/9731d2fb17bf425c99b22f165294d113 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:49.445076 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693a4a8-f7e5-4e37-ac0a-8d023ee8d519 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:49.457232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12da8c-473a-42c8-ba69-5e0b832fbb25 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:49.499406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f0037-75d6-4ea3-994d-ae046f77160a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:49.500762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f0037-75d6-4ea3-994d-ae046f77160a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:49.501619 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f0037-75d6-4ea3-994d-ae046f77160a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:49.502106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f0037-75d6-4ea3-994d-ae046f77160a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:49.503345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f0037-75d6-4ea3-994d-ae046f77160a None admin] RBAC: Adding query filter params (role.id=e385f7d11fac4932a910ec0ef95e2a43, user.id=0da6c88bcdaa465dbf10d7c77465239c, scope.system=all) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:49.504347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f0037-75d6-4ea3-994d-ae046f77160a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:49.505996 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f0037-75d6-4ea3-994d-ae046f77160a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:49.522832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 166/333] 10.222.0.62 () {56 vars in 1308 bytes} [Fri Sep 9 11:52:49 2022] GET /identity/v3/role_assignments?user.id=0da6c88bcdaa465dbf10d7c77465239c&scope.system=all&role.id=e385f7d11fac4932a910ec0ef95e2a43 => generated 510 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:49.579805 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:49.583802 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693a4a8-f7e5-4e37-ac0a-8d023ee8d519 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:49.623878 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19090ab8-52c4-4be4-afb8-5a5b32140416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:49.624551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19090ab8-52c4-4be4-afb8-5a5b32140416 None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:49.624832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19090ab8-52c4-4be4-afb8-5a5b32140416 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:49.625102 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19090ab8-52c4-4be4-afb8-5a5b32140416 None admin] PATH_INFO: `/v3/system/users/9731d2fb17bf425c99b22f165294d113/roles/80400d1e5fdb44b3a92a468f6b62b248` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:49.626607 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19090ab8-52c4-4be4-afb8-5a5b32140416 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9731d2fb17bf425c99b22f165294d113, role_id=80400d1e5fdb44b3a92a468f6b62b248)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:49.628089 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19090ab8-52c4-4be4-afb8-5a5b32140416 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:49.639469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 168/334] 10.222.0.62 () {58 vars in 1334 bytes} [Fri Sep 9 11:52:49 2022] PUT /identity/v3/system/users/9731d2fb17bf425c99b22f165294d113/roles/80400d1e5fdb44b3a92a468f6b62b248 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:52:51.503871 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262197e8-ebe7-463c-b95c-6b9d1faa611c None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:51.504375 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262197e8-ebe7-463c-b95c-6b9d1faa611c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:51.504375 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262197e8-ebe7-463c-b95c-6b9d1faa611c None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:51.505904 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 167/335] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:51.512194 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfe2f3-7355-4949-bd4e-009b81d256e8 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:51.512655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfe2f3-7355-4949-bd4e-009b81d256e8 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:51.512729 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfe2f3-7355-4949-bd4e-009b81d256e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:51.533556 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-52cfe2f3-7355-4949-bd4e-009b81d256e8 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:51.534768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-52cfe2f3-7355-4949-bd4e-009b81d256e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:51.627089 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 169/336] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:51.700393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948635c-7d9c-471b-bf15-fdd56302b1bf None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:51.700718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948635c-7d9c-471b-bf15-fdd56302b1bf None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:51.700936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948635c-7d9c-471b-bf15-fdd56302b1bf None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:51.702570 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 168/337] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:51.724189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9118e-3f68-412f-8073-a7c1076b8861 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:51.724802 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9118e-3f68-412f-8073-a7c1076b8861 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:51.725273 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9118e-3f68-412f-8073-a7c1076b8861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:51.746535 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f5a9118e-3f68-412f-8073-a7c1076b8861 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:51.748046 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f5a9118e-3f68-412f-8073-a7c1076b8861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:51.831434 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 170/338] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:52.271628 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948635c-7d9c-471b-bf15-fdd56302b1bf None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:52.304005 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b1f1d-71f5-4f2b-8cd4-8876756a0025 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:52.304639 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b1f1d-71f5-4f2b-8cd4-8876756a0025 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:52.305673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b1f1d-71f5-4f2b-8cd4-8876756a0025 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:52.308549 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 171/339] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:52.312832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83553ec1-3783-4a0e-8b66-f48333232912 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:52.313350 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83553ec1-3783-4a0e-8b66-f48333232912 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:52.313839 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83553ec1-3783-4a0e-8b66-f48333232912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:52.351135 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-83553ec1-3783-4a0e-8b66-f48333232912 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:52.352763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-83553ec1-3783-4a0e-8b66-f48333232912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:52.394144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:52.407224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948635c-7d9c-471b-bf15-fdd56302b1bf None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:52.458552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:52.460536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461592e7-3129-4f01-8083-9e2d18929b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:52.462039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461592e7-3129-4f01-8083-9e2d18929b03 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:52.462501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461592e7-3129-4f01-8083-9e2d18929b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:52.462920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461592e7-3129-4f01-8083-9e2d18929b03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:52.480186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461592e7-3129-4f01-8083-9e2d18929b03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-461592e7-3129-4f01-8083-9e2d18929b03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:52.488426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:52.489566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:52.490590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:52.490590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:52.490590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:52.490590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:52.490590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:52.490590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:52.490590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:52.490590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:52.490590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:52.490590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:52.490590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:52.490590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:52.493345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 169/340] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:52.494426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461592e7-3129-4f01-8083-9e2d18929b03 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:52.502156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 172/341] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:52.503024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83553ec1-3783-4a0e-8b66-f48333232912 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:52.666146 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83553ec1-3783-4a0e-8b66-f48333232912 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:52.671417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461592e7-3129-4f01-8083-9e2d18929b03 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:52.728105 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40117ff1-bbbd-4c14-b2f8-c1fff3c524e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:52.728734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40117ff1-bbbd-4c14-b2f8-c1fff3c524e6 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:52.729016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40117ff1-bbbd-4c14-b2f8-c1fff3c524e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:52.729290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40117ff1-bbbd-4c14-b2f8-c1fff3c524e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:52.729509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257b8b5-f4bb-4b1c-b5df-4e33e3878f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:52.729968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40117ff1-bbbd-4c14-b2f8-c1fff3c524e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:52.730350 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40117ff1-bbbd-4c14-b2f8-c1fff3c524e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:52.730855 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257b8b5-f4bb-4b1c-b5df-4e33e3878f32 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:52.731337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257b8b5-f4bb-4b1c-b5df-4e33e3878f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:52.731680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40117ff1-bbbd-4c14-b2f8-c1fff3c524e6 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:52.732001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257b8b5-f4bb-4b1c-b5df-4e33e3878f32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:52.744403 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 173/342] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:52.747458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40117ff1-bbbd-4c14-b2f8-c1fff3c524e6 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:52.750456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257b8b5-f4bb-4b1c-b5df-4e33e3878f32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-8257b8b5-f4bb-4b1c-b5df-4e33e3878f32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:52.755910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:52.756779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:52.757571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:52.757571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:52.757571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:52.757571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:52.757571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:52.757571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:52.757571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:52.757571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:52.757571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:52.757571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:52.757571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:52.757571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:52.760320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 170/343] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:52:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:52.766036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257b8b5-f4bb-4b1c-b5df-4e33e3878f32 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:52.771503 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:52.834721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:52.899789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40117ff1-bbbd-4c14-b2f8-c1fff3c524e6 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:52.934534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257b8b5-f4bb-4b1c-b5df-4e33e3878f32 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:52.959392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeec59d-ad4c-4516-9ce3-22fc8b66f612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:52.960262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeec59d-ad4c-4516-9ce3-22fc8b66f612 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:52.960718 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeec59d-ad4c-4516-9ce3-22fc8b66f612 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:52.961145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeec59d-ad4c-4516-9ce3-22fc8b66f612 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:52.991903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453d0de-e310-4462-bfca-7397d2015311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:52.993277 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453d0de-e310-4462-bfca-7397d2015311 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:52.993695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453d0de-e310-4462-bfca-7397d2015311 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:52.994143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453d0de-e310-4462-bfca-7397d2015311 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:52.994963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453d0de-e310-4462-bfca-7397d2015311 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:52.995422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453d0de-e310-4462-bfca-7397d2015311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:52.997012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453d0de-e310-4462-bfca-7397d2015311 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:53.011533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 171/344] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:52:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:53.013019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23244bc7-9b7f-4786-ada4-f2877fa82038 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:53.013443 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23244bc7-9b7f-4786-ada4-f2877fa82038 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:53.013856 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23244bc7-9b7f-4786-ada4-f2877fa82038 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:53.015264 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 172/345] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:53.019000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23244bc7-9b7f-4786-ada4-f2877fa82038 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:53.046983 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeec59d-ad4c-4516-9ce3-22fc8b66f612 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:53.048706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeec59d-ad4c-4516-9ce3-22fc8b66f612 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:53.070605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:53.194444 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:53.203834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23244bc7-9b7f-4786-ada4-f2877fa82038 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:53.239193 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 174/346] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:52:53.240865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07556f13-3755-4289-9814-7eca293d71ca None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:53.241154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07556f13-3755-4289-9814-7eca293d71ca None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:53.241427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07556f13-3755-4289-9814-7eca293d71ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:53.241537 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b14b1-ebfc-456b-846c-bf6d6cbf1585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:53.242097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b14b1-ebfc-456b-846c-bf6d6cbf1585 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:53.242673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b14b1-ebfc-456b-846c-bf6d6cbf1585 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:53.243129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b14b1-ebfc-456b-846c-bf6d6cbf1585 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:53.263078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-07556f13-3755-4289-9814-7eca293d71ca None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:53.264411 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-07556f13-3755-4289-9814-7eca293d71ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:53.325154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b14b1-ebfc-456b-846c-bf6d6cbf1585 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:53.326952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b14b1-ebfc-456b-846c-bf6d6cbf1585 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:53.343785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 175/347] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:53.344960 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07556f13-3755-4289-9814-7eca293d71ca None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:53.397453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:53.444333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:53.451214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 173/348] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:53.452658 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b14b1-ebfc-456b-846c-bf6d6cbf1585 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:53.469304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07556f13-3755-4289-9814-7eca293d71ca None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:53.507098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086fb17-0448-41e0-a25b-2bca65c95d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:53.507970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086fb17-0448-41e0-a25b-2bca65c95d70 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:53.508438 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086fb17-0448-41e0-a25b-2bca65c95d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:53.508863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086fb17-0448-41e0-a25b-2bca65c95d70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:53.520299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1086fb17-0448-41e0-a25b-2bca65c95d70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-1086fb17-0448-41e0-a25b-2bca65c95d70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:53.525818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:53.527009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:53.528068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:53.528068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:53.528068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:53.528068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:53.528068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:53.528068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:53.528068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:53.528068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:53.528068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:53.528068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:53.528068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:53.528068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:53.529971 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 176/349] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:53.530921 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086fb17-0448-41e0-a25b-2bca65c95d70 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:53.645653 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086fb17-0448-41e0-a25b-2bca65c95d70 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:53.650461 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:53.681513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b14b1-ebfc-456b-846c-bf6d6cbf1585 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:53.692238 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6caf8d-76d4-4125-9ddc-da07e6899f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:53.693506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6caf8d-76d4-4125-9ddc-da07e6899f69 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:53.693942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6caf8d-76d4-4125-9ddc-da07e6899f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:53.694381 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6caf8d-76d4-4125-9ddc-da07e6899f69 None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:53.696467 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6caf8d-76d4-4125-9ddc-da07e6899f69 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:53.698822 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6caf8d-76d4-4125-9ddc-da07e6899f69 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:53.701117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 177/350] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:53 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:53.702150 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6caf8d-76d4-4125-9ddc-da07e6899f69 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:53.790287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268a4be-29b5-4aa3-a9ba-4ef1e6988011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:53.791298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268a4be-29b5-4aa3-a9ba-4ef1e6988011 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:53.792231 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268a4be-29b5-4aa3-a9ba-4ef1e6988011 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:53.792704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268a4be-29b5-4aa3-a9ba-4ef1e6988011 None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:53.794820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268a4be-29b5-4aa3-a9ba-4ef1e6988011 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:53.796877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268a4be-29b5-4aa3-a9ba-4ef1e6988011 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:53.799965 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 174/351] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:53 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:53.801752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268a4be-29b5-4aa3-a9ba-4ef1e6988011 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:53.818426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6caf8d-76d4-4125-9ddc-da07e6899f69 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:53.864814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353a3ed-b8c7-48a7-b973-eeb2952d81cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:53.865761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353a3ed-b8c7-48a7-b973-eeb2952d81cd None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:53.866219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353a3ed-b8c7-48a7-b973-eeb2952d81cd None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:53.866492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353a3ed-b8c7-48a7-b973-eeb2952d81cd None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:53.867168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353a3ed-b8c7-48a7-b973-eeb2952d81cd None admin] RBAC: Adding query filter params (name=member) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:53.868041 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353a3ed-b8c7-48a7-b973-eeb2952d81cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:53.869489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353a3ed-b8c7-48a7-b973-eeb2952d81cd None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:53.878832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 178/352] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:53.879519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353a3ed-b8c7-48a7-b973-eeb2952d81cd None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:53.884801 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:53.920559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:53.941324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268a4be-29b5-4aa3-a9ba-4ef1e6988011 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:53.992031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c4d42-5aae-41b5-af1c-2334e741c0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:53.993540 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c4d42-5aae-41b5-af1c-2334e741c0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:53.994009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c4d42-5aae-41b5-af1c-2334e741c0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:53.994320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c4d42-5aae-41b5-af1c-2334e741c0b9 None admin] PATH_INFO: `/v3/groups/ce9fe4f82f1f43e3be83457d040809b3` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:54.002479 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c4d42-5aae-41b5-af1c-2334e741c0b9 None admin] RBAC: Authorizing `identity:get_group(group_id=ce9fe4f82f1f43e3be83457d040809b3)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:54.003916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c4d42-5aae-41b5-af1c-2334e741c0b9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:54.005847 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 175/353] 10.222.0.62 () {56 vars in 1180 bytes} [Fri Sep 9 11:52:53 2022] GET /identity/v3/groups/ce9fe4f82f1f43e3be83457d040809b3 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:54.005967 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353a3ed-b8c7-48a7-b973-eeb2952d81cd None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:54.006635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c4d42-5aae-41b5-af1c-2334e741c0b9 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:54.053385 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024394f-02a9-43a0-b66b-65dc52be633b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:54.054772 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024394f-02a9-43a0-b66b-65dc52be633b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:54.055783 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024394f-02a9-43a0-b66b-65dc52be633b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:54.056211 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024394f-02a9-43a0-b66b-65dc52be633b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:54.071332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024394f-02a9-43a0-b66b-65dc52be633b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-6024394f-02a9-43a0-b66b-65dc52be633b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:54.077109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:54.079034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:54.079905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:54.079905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:54.079905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:54.079905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:54.079905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:54.079905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:54.079905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:54.079905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:54.079905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:54.079905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:54.079905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:52:54.079905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:54.082090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 179/354] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:54.083588 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b844bb3-4a5f-4b80-97f2-5e6134048ba2 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:54.084144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b844bb3-4a5f-4b80-97f2-5e6134048ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:54.084597 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b844bb3-4a5f-4b80-97f2-5e6134048ba2 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:54.086222 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 180/355] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:54.087801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b844bb3-4a5f-4b80-97f2-5e6134048ba2 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:54.140306 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c4d42-5aae-41b5-af1c-2334e741c0b9 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:54.184737 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:54.201460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f046b5-ffa2-4d70-a0c7-18ec106c43c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:54.202199 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f046b5-ffa2-4d70-a0c7-18ec106c43c0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:54.202456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f046b5-ffa2-4d70-a0c7-18ec106c43c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:54.202739 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f046b5-ffa2-4d70-a0c7-18ec106c43c0 None admin] PATH_INFO: `/v3/groups/ce9fe4f82f1f43e3be83457d040809b3` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:54.204638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f046b5-ffa2-4d70-a0c7-18ec106c43c0 None admin] RBAC: Authorizing `identity:get_group(group_id=ce9fe4f82f1f43e3be83457d040809b3)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:54.206087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f046b5-ffa2-4d70-a0c7-18ec106c43c0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:54.207724 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 176/356] 10.222.0.62 () {56 vars in 1180 bytes} [Fri Sep 9 11:52:54 2022] GET /identity/v3/groups/ce9fe4f82f1f43e3be83457d040809b3 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:54.210761 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f046b5-ffa2-4d70-a0c7-18ec106c43c0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:54.213869 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:54.224791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b844bb3-4a5f-4b80-97f2-5e6134048ba2 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:54.270903 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20b629-7987-49de-af94-1d87e541593b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:54.271524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20b629-7987-49de-af94-1d87e541593b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:54.271799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20b629-7987-49de-af94-1d87e541593b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:54.272402 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20b629-7987-49de-af94-1d87e541593b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:54.335800 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f046b5-ffa2-4d70-a0c7-18ec106c43c0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:54.350704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b20b629-7987-49de-af94-1d87e541593b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:54.352966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b20b629-7987-49de-af94-1d87e541593b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:54.381164 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a2396-f11b-4eaf-8ee6-85af99b95805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:54.381164 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a2396-f11b-4eaf-8ee6-85af99b95805 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:54.381164 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a2396-f11b-4eaf-8ee6-85af99b95805 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:54.381510 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a2396-f11b-4eaf-8ee6-85af99b95805 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:54.382032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a2396-f11b-4eaf-8ee6-85af99b95805 None admin] RBAC: Adding query filter params (group.id=ce9fe4f82f1f43e3be83457d040809b3, role.id=d43148a29472443ba7dfe2424ec60baf, scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:54.383271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a2396-f11b-4eaf-8ee6-85af99b95805 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:54.384704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a2396-f11b-4eaf-8ee6-85af99b95805 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:54.390510 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 177/357] 10.222.0.62 () {56 vars in 1376 bytes} [Fri Sep 9 11:52:54 2022] GET /identity/v3/role_assignments?group.id=ce9fe4f82f1f43e3be83457d040809b3&scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa&role.id=d43148a29472443ba7dfe2424ec60baf => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:54.392551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28379296-d7eb-4379-9806-e588ed673d63 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:54.392835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28379296-d7eb-4379-9806-e588ed673d63 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:54.393107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28379296-d7eb-4379-9806-e588ed673d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:54.419042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-28379296-d7eb-4379-9806-e588ed673d63 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:54.420720 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-28379296-d7eb-4379-9806-e588ed673d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:54.471799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:54.484382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 181/358] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:54.484382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20b629-7987-49de-af94-1d87e541593b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:54.518168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:54.536781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 178/359] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:54.538238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28379296-d7eb-4379-9806-e588ed673d63 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:54.643714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20b629-7987-49de-af94-1d87e541593b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:54.681448 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163a42e-b48c-4d53-9513-edaef3009ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:54.682233 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163a42e-b48c-4d53-9513-edaef3009ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:54.682574 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163a42e-b48c-4d53-9513-edaef3009ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:54.682919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163a42e-b48c-4d53-9513-edaef3009ef7 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:54.683597 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163a42e-b48c-4d53-9513-edaef3009ef7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:54.683900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163a42e-b48c-4d53-9513-edaef3009ef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:54.685289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163a42e-b48c-4d53-9513-edaef3009ef7 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:54.685502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28379296-d7eb-4379-9806-e588ed673d63 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:54.695110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 182/360] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:54.696962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68138553-8f0f-4ed1-af10-092cc917a0b1 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:54.697243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68138553-8f0f-4ed1-af10-092cc917a0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:54.697507 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68138553-8f0f-4ed1-af10-092cc917a0b1 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:54.698852 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 183/361] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:54.699621 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68138553-8f0f-4ed1-af10-092cc917a0b1 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:54.736825 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60e2a0-adb3-4f12-8b87-fbc5a60e9bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:54.737447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60e2a0-adb3-4f12-8b87-fbc5a60e9bda None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:54.737726 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60e2a0-adb3-4f12-8b87-fbc5a60e9bda None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:54.737996 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60e2a0-adb3-4f12-8b87-fbc5a60e9bda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:54.739031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60e2a0-adb3-4f12-8b87-fbc5a60e9bda None admin] RBAC: Adding query filter params (group.id=ce9fe4f82f1f43e3be83457d040809b3, role.id=e385f7d11fac4932a910ec0ef95e2a43, scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:54.739766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60e2a0-adb3-4f12-8b87-fbc5a60e9bda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:54.741701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60e2a0-adb3-4f12-8b87-fbc5a60e9bda None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:54.749085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 179/362] 10.222.0.62 () {56 vars in 1376 bytes} [Fri Sep 9 11:52:54 2022] GET /identity/v3/role_assignments?group.id=ce9fe4f82f1f43e3be83457d040809b3&scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa&role.id=e385f7d11fac4932a910ec0ef95e2a43 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:54.749924 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60e2a0-adb3-4f12-8b87-fbc5a60e9bda None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:54.790335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:54.886168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:54.891727 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68138553-8f0f-4ed1-af10-092cc917a0b1 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:54.948073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a79a8-609c-4f40-8236-1149332c65b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:54.948637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a79a8-609c-4f40-8236-1149332c65b8 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:54.948911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a79a8-609c-4f40-8236-1149332c65b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:54.949176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a79a8-609c-4f40-8236-1149332c65b8 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:54.950509 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a79a8-609c-4f40-8236-1149332c65b8 None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:54.951955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a79a8-609c-4f40-8236-1149332c65b8 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:54.953807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 184/363] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:54 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:54.956272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c0e5d-41bc-40c7-9e04-f4c184e45052 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:54.956272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c0e5d-41bc-40c7-9e04-f4c184e45052 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:54.956272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c0e5d-41bc-40c7-9e04-f4c184e45052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:54.959933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60e2a0-adb3-4f12-8b87-fbc5a60e9bda None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:54.983346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-df6c0e5d-41bc-40c7-9e04-f4c184e45052 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:54.985274 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-df6c0e5d-41bc-40c7-9e04-f4c184e45052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:55.015468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b527c-5f12-4a6e-a598-4835c8cb48dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:55.016119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b527c-5f12-4a6e-a598-4835c8cb48dd None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:55.016399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b527c-5f12-4a6e-a598-4835c8cb48dd None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:55.016673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b527c-5f12-4a6e-a598-4835c8cb48dd None admin] PATH_INFO: `/v3/users/9731d2fb17bf425c99b22f165294d113` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:55.018621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b527c-5f12-4a6e-a598-4835c8cb48dd None admin] RBAC: Authorizing `identity:get_user(user_id=9731d2fb17bf425c99b22f165294d113)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:55.020090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b527c-5f12-4a6e-a598-4835c8cb48dd None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:55.021693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 180/364] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:52:54 2022] GET /identity/v3/users/9731d2fb17bf425c99b22f165294d113 => generated 293 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:55.022673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b527c-5f12-4a6e-a598-4835c8cb48dd None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:55.072641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 185/365] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:55.073577 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c0e5d-41bc-40c7-9e04-f4c184e45052 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:55.141240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:55.142778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b527c-5f12-4a6e-a598-4835c8cb48dd None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:55.195333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:55.208299 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39ebe0-dc7c-47e7-9359-209aa8be773c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:55.208866 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39ebe0-dc7c-47e7-9359-209aa8be773c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:55.209135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39ebe0-dc7c-47e7-9359-209aa8be773c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:55.209400 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39ebe0-dc7c-47e7-9359-209aa8be773c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:55.229433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39ebe0-dc7c-47e7-9359-209aa8be773c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-3b39ebe0-dc7c-47e7-9359-209aa8be773c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:55.235357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:55.236279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:55.237072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:55.237072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:55.237072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:55.237072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:55.237072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:55.237072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:55.237072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:55.237072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:55.237072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:55.237072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:55.237072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:55.237072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:55.238441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c0e5d-41bc-40c7-9e04-f4c184e45052 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:55.245653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 181/366] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:52:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:55.246643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39ebe0-dc7c-47e7-9359-209aa8be773c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:55.304033 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6363f96-4fad-4560-b6cd-9f110bf38c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:55.304676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6363f96-4fad-4560-b6cd-9f110bf38c9d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:55.304982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6363f96-4fad-4560-b6cd-9f110bf38c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:55.305246 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6363f96-4fad-4560-b6cd-9f110bf38c9d None admin] PATH_INFO: `/v3/groups/ce9fe4f82f1f43e3be83457d040809b3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:55.308187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6363f96-4fad-4560-b6cd-9f110bf38c9d None admin] RBAC: Authorizing `identity:get_group(group_id=ce9fe4f82f1f43e3be83457d040809b3)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:55.309610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6363f96-4fad-4560-b6cd-9f110bf38c9d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:55.311425 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 186/367] 10.222.0.62 () {56 vars in 1180 bytes} [Fri Sep 9 11:52:55 2022] GET /identity/v3/groups/ce9fe4f82f1f43e3be83457d040809b3 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:55.311962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6363f96-4fad-4560-b6cd-9f110bf38c9d None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:55.426325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39ebe0-dc7c-47e7-9359-209aa8be773c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:55.477818 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f02854-a5db-4138-9d5b-927436246c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:55.478387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f02854-a5db-4138-9d5b-927436246c9e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:55.478935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f02854-a5db-4138-9d5b-927436246c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:55.479205 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f02854-a5db-4138-9d5b-927436246c9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:55.479940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f02854-a5db-4138-9d5b-927436246c9e None admin] RBAC: Adding query filter params (role.id=80400d1e5fdb44b3a92a468f6b62b248, user.id=9731d2fb17bf425c99b22f165294d113, scope.system=all) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:55.480360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f02854-a5db-4138-9d5b-927436246c9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:55.481906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f02854-a5db-4138-9d5b-927436246c9e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:55.492733 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6363f96-4fad-4560-b6cd-9f110bf38c9d None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:55.500072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 182/368] 10.222.0.62 () {56 vars in 1308 bytes} [Fri Sep 9 11:52:55 2022] GET /identity/v3/role_assignments?user.id=9731d2fb17bf425c99b22f165294d113&scope.system=all&role.id=80400d1e5fdb44b3a92a468f6b62b248 => generated 510 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:55.500975 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f02854-a5db-4138-9d5b-927436246c9e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:55.539127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:55.563560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff9837-10de-4d91-9db8-83552869e0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:55.564187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff9837-10de-4d91-9db8-83552869e0bd None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:55.564467 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff9837-10de-4d91-9db8-83552869e0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:55.564749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff9837-10de-4d91-9db8-83552869e0bd None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:55.565408 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff9837-10de-4d91-9db8-83552869e0bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:55.565708 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff9837-10de-4d91-9db8-83552869e0bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:55.567128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff9837-10de-4d91-9db8-83552869e0bd None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:55.589044 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 187/369] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:52:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:55.589696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff9837-10de-4d91-9db8-83552869e0bd None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:55.601886 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:55.711901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f02854-a5db-4138-9d5b-927436246c9e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:55.757394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7974b1-d379-40e2-a275-9bd23ec9705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:55.758683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7974b1-d379-40e2-a275-9bd23ec9705b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:55.758968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7974b1-d379-40e2-a275-9bd23ec9705b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:55.759240 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7974b1-d379-40e2-a275-9bd23ec9705b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:55.759988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7974b1-d379-40e2-a275-9bd23ec9705b None admin] RBAC: Adding query filter params (group.id=ce9fe4f82f1f43e3be83457d040809b3, role.id=e385f7d11fac4932a910ec0ef95e2a43, scope.project.id=3a96e4f9e7134925af6e684f79748f71) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:55.760295 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7974b1-d379-40e2-a275-9bd23ec9705b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:55.762160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7974b1-d379-40e2-a275-9bd23ec9705b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:55.769082 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 183/370] 10.222.0.62 () {56 vars in 1376 bytes} [Fri Sep 9 11:52:55 2022] GET /identity/v3/role_assignments?group.id=ce9fe4f82f1f43e3be83457d040809b3&scope.project.id=3a96e4f9e7134925af6e684f79748f71&role.id=e385f7d11fac4932a910ec0ef95e2a43 => generated 260 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:55.769955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7974b1-d379-40e2-a275-9bd23ec9705b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:55.805691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff9837-10de-4d91-9db8-83552869e0bd None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:55.883820 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5392bd-d5d1-4870-886f-6c02f955d788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:55.884486 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5392bd-d5d1-4870-886f-6c02f955d788 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:55.884762 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5392bd-d5d1-4870-886f-6c02f955d788 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:55.885037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5392bd-d5d1-4870-886f-6c02f955d788 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:55.913415 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5392bd-d5d1-4870-886f-6c02f955d788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-1b5392bd-d5d1-4870-886f-6c02f955d788 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:55.921896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:55.922970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:55.924751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:55.924751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:55.924751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:55.924751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:55.924751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:55.924751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:55.924751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:55.924751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:55.924751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:55.924751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:55.924751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:52:55.924751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:55.924751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 188/371] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:52:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:55.930081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5392bd-d5d1-4870-886f-6c02f955d788 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:55.970195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7974b1-d379-40e2-a275-9bd23ec9705b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:55.988397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:55.989663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:56.053770 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1cf8e-17fd-4656-b037-c52dd4f82ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:56.054342 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1cf8e-17fd-4656-b037-c52dd4f82ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:56.054889 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1cf8e-17fd-4656-b037-c52dd4f82ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:56.055162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1cf8e-17fd-4656-b037-c52dd4f82ed0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:56.062958 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5392bd-d5d1-4870-886f-6c02f955d788 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:56.109913 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ad830-4d7d-4785-9628-79e5f385d593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:56.110710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ad830-4d7d-4785-9628-79e5f385d593 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:56.110985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ad830-4d7d-4785-9628-79e5f385d593 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:56.111564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ad830-4d7d-4785-9628-79e5f385d593 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:56.112505 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ad830-4d7d-4785-9628-79e5f385d593 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:56.112804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ad830-4d7d-4785-9628-79e5f385d593 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:56.114409 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ad830-4d7d-4785-9628-79e5f385d593 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:56.125670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 189/372] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:56.131804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ad830-4d7d-4785-9628-79e5f385d593 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:56.159280 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1cf8e-17fd-4656-b037-c52dd4f82ed0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:56.160738 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1cf8e-17fd-4656-b037-c52dd4f82ed0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:56.263529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ad830-4d7d-4785-9628-79e5f385d593 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:56.272251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:56.284416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 184/373] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:56.298570 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1cf8e-17fd-4656-b037-c52dd4f82ed0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:56.314229 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2eecb6-b3b5-4182-b54e-fe999dee723a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:56.314229 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eecb6-b3b5-4182-b54e-fe999dee723a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:56.314229 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eecb6-b3b5-4182-b54e-fe999dee723a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:56.314229 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eecb6-b3b5-4182-b54e-fe999dee723a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:56.316976 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:56.398142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2eecb6-b3b5-4182-b54e-fe999dee723a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:56.398142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2eecb6-b3b5-4182-b54e-fe999dee723a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:56.529832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1cf8e-17fd-4656-b037-c52dd4f82ed0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:56.554060 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 190/374] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:52:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:52:56.560546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2eecb6-b3b5-4182-b54e-fe999dee723a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:56.573345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:56.599645 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45f355-4527-4294-b8a3-3fb06c558db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:56.600293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45f355-4527-4294-b8a3-3fb06c558db2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:56.600567 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45f355-4527-4294-b8a3-3fb06c558db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:56.600840 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45f355-4527-4294-b8a3-3fb06c558db2 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:56.603643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45f355-4527-4294-b8a3-3fb06c558db2 None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:56.605082 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45f355-4527-4294-b8a3-3fb06c558db2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:56.612763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 185/375] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:56 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:56.613634 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45f355-4527-4294-b8a3-3fb06c558db2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:56.728847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2eecb6-b3b5-4182-b54e-fe999dee723a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:56.741657 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:56.766488 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993368d-b621-4d64-852b-a2aeb3446dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:56.767464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993368d-b621-4d64-852b-a2aeb3446dbe None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:56.767886 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993368d-b621-4d64-852b-a2aeb3446dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:56.768314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993368d-b621-4d64-852b-a2aeb3446dbe None admin] PATH_INFO: `/v3/projects/fe7d1fa6c42548518e72832336a573e5` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:56.769683 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993368d-b621-4d64-852b-a2aeb3446dbe None admin] RBAC: Authorizing `identity:get_project(project_id=fe7d1fa6c42548518e72832336a573e5)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:56.771346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993368d-b621-4d64-852b-a2aeb3446dbe None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:56.774203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 191/376] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:56 2022] GET /identity/v3/projects/fe7d1fa6c42548518e72832336a573e5 => generated 344 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:56.776172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45f355-4527-4294-b8a3-3fb06c558db2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:56.783195 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993368d-b621-4d64-852b-a2aeb3446dbe None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:56.841902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35d9e2-cf35-4842-bb0b-f36d851e03bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:56.842583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35d9e2-cf35-4842-bb0b-f36d851e03bc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:56.842858 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35d9e2-cf35-4842-bb0b-f36d851e03bc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:56.843128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35d9e2-cf35-4842-bb0b-f36d851e03bc None admin] PATH_INFO: `/v3/groups/ce9fe4f82f1f43e3be83457d040809b3` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:56.846193 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35d9e2-cf35-4842-bb0b-f36d851e03bc None admin] RBAC: Authorizing `identity:get_group(group_id=ce9fe4f82f1f43e3be83457d040809b3)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:56.847655 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35d9e2-cf35-4842-bb0b-f36d851e03bc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:56.849443 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 186/377] 10.222.0.62 () {56 vars in 1180 bytes} [Fri Sep 9 11:52:56 2022] GET /identity/v3/groups/ce9fe4f82f1f43e3be83457d040809b3 => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:56.863121 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35d9e2-cf35-4842-bb0b-f36d851e03bc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:56.933845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:56.987652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993368d-b621-4d64-852b-a2aeb3446dbe None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:57.061913 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af320a-9f6c-4e8a-812c-3b1b12f85220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:57.062877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af320a-9f6c-4e8a-812c-3b1b12f85220 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:57.063158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af320a-9f6c-4e8a-812c-3b1b12f85220 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:57.063421 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af320a-9f6c-4e8a-812c-3b1b12f85220 None admin] PATH_INFO: `/v3/groups/5deb34b4087e46c4a4cbe16c7fe15338` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:57.065684 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35d9e2-cf35-4842-bb0b-f36d851e03bc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:57.080116 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af320a-9f6c-4e8a-812c-3b1b12f85220 None admin] RBAC: Authorizing `identity:get_group(group_id=5deb34b4087e46c4a4cbe16c7fe15338)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:57.081654 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af320a-9f6c-4e8a-812c-3b1b12f85220 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:57.083578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 192/378] 10.222.0.62 () {56 vars in 1180 bytes} [Fri Sep 9 11:52:56 2022] GET /identity/v3/groups/5deb34b4087e46c4a4cbe16c7fe15338 => generated 229 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:57.089725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af320a-9f6c-4e8a-812c-3b1b12f85220 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:57.136039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3732d3-f5c8-471b-a14a-07c599420052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:57.136566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3732d3-f5c8-471b-a14a-07c599420052 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:57.136902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3732d3-f5c8-471b-a14a-07c599420052 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:57.137172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3732d3-f5c8-471b-a14a-07c599420052 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:57.137898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3732d3-f5c8-471b-a14a-07c599420052 None admin] RBAC: Adding query filter params (group.id=ce9fe4f82f1f43e3be83457d040809b3, role.id=d43148a29472443ba7dfe2424ec60baf, scope.project.id=3a96e4f9e7134925af6e684f79748f71) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:57.138221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3732d3-f5c8-471b-a14a-07c599420052 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:57.140353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3732d3-f5c8-471b-a14a-07c599420052 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:57.151198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 187/379] 10.222.0.62 () {56 vars in 1376 bytes} [Fri Sep 9 11:52:56 2022] GET /identity/v3/role_assignments?group.id=ce9fe4f82f1f43e3be83457d040809b3&scope.project.id=3a96e4f9e7134925af6e684f79748f71&role.id=d43148a29472443ba7dfe2424ec60baf => generated 260 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:57.258995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af320a-9f6c-4e8a-812c-3b1b12f85220 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:57.290580 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71994f84-1430-463d-80aa-fb8f0e0dcec5 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:57.290912 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71994f84-1430-463d-80aa-fb8f0e0dcec5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:57.291112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71994f84-1430-463d-80aa-fb8f0e0dcec5 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:57.293048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 188/380] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:52:57.309485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e38dd-1596-4035-8967-ecf08490233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:57.309693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0807d-d32f-4954-8f1e-97c7d8c09090 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:57.309952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0807d-d32f-4954-8f1e-97c7d8c09090 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:57.310079 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e38dd-1596-4035-8967-ecf08490233d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:57.310274 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0807d-d32f-4954-8f1e-97c7d8c09090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:57.310370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e38dd-1596-4035-8967-ecf08490233d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:57.310636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e38dd-1596-4035-8967-ecf08490233d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:57.311366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e38dd-1596-4035-8967-ecf08490233d None admin] RBAC: Adding query filter params (group.id=5deb34b4087e46c4a4cbe16c7fe15338, role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=fe7d1fa6c42548518e72832336a573e5) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:57.311669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e38dd-1596-4035-8967-ecf08490233d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:57.313329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e38dd-1596-4035-8967-ecf08490233d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:57.319838 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 193/381] 10.222.0.62 () {56 vars in 1376 bytes} [Fri Sep 9 11:52:57 2022] GET /identity/v3/role_assignments?group.id=5deb34b4087e46c4a4cbe16c7fe15338&scope.project.id=fe7d1fa6c42548518e72832336a573e5&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:57.344253 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5ab0807d-d32f-4954-8f1e-97c7d8c09090 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:57.345490 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5ab0807d-d32f-4954-8f1e-97c7d8c09090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:57.347828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:57.445185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 189/382] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:57.728209 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b467c6-375d-490f-9782-4027848a5bbd None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:57.728550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b467c6-375d-490f-9782-4027848a5bbd None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:57.728663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b467c6-375d-490f-9782-4027848a5bbd None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:57.730264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 194/383] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:57.736098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a15b3c-43d8-4759-83a0-9a8a9dd1e4f8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:57.736676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a15b3c-43d8-4759-83a0-9a8a9dd1e4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:57.737143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a15b3c-43d8-4759-83a0-9a8a9dd1e4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:57.759545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f7a15b3c-43d8-4759-83a0-9a8a9dd1e4f8 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:57.761504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f7a15b3c-43d8-4759-83a0-9a8a9dd1e4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:57.810186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b467c6-375d-490f-9782-4027848a5bbd None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:57.813527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:57.855007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 190/384] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:52:57.954633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b467c6-375d-490f-9782-4027848a5bbd None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:57.997758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089467af-edd4-4db9-ba8d-b632328cfbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:57.998399 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089467af-edd4-4db9-ba8d-b632328cfbb7 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:57.998680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089467af-edd4-4db9-ba8d-b632328cfbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:57.998949 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089467af-edd4-4db9-ba8d-b632328cfbb7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:58.011677 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089467af-edd4-4db9-ba8d-b632328cfbb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-089467af-edd4-4db9-ba8d-b632328cfbb7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:58.017078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:58.017974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:58.018804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:58.018804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:58.018804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:58.018804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:58.018804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:58.018804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:58.018804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:58.018804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:58.018804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:58.018804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:58.018804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:52:58.018804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:58.018804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 195/385] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:52:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:52:58.024232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a15b3c-43d8-4759-83a0-9a8a9dd1e4f8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:58.097602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:58.134036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a15b3c-43d8-4759-83a0-9a8a9dd1e4f8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:58.171638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882443c-e3f3-4d6e-8449-10b685c1cfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:58.172255 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882443c-e3f3-4d6e-8449-10b685c1cfa4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:58.172529 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882443c-e3f3-4d6e-8449-10b685c1cfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:58.172798 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882443c-e3f3-4d6e-8449-10b685c1cfa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:58.173469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882443c-e3f3-4d6e-8449-10b685c1cfa4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:58.173831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882443c-e3f3-4d6e-8449-10b685c1cfa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:58.175225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882443c-e3f3-4d6e-8449-10b685c1cfa4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:58.185093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 191/386] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:52:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:58.190650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089467af-edd4-4db9-ba8d-b632328cfbb7 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:58.275683 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:58.319363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089467af-edd4-4db9-ba8d-b632328cfbb7 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:58.364691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972e413-44f4-456f-a7d2-def4f7e24c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:58.365342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972e413-44f4-456f-a7d2-def4f7e24c8d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:58.365619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972e413-44f4-456f-a7d2-def4f7e24c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:58.365957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972e413-44f4-456f-a7d2-def4f7e24c8d None admin] PATH_INFO: `/v3/groups/ce9fe4f82f1f43e3be83457d040809b3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:58.367124 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972e413-44f4-456f-a7d2-def4f7e24c8d None admin] RBAC: Authorizing `identity:get_group(group_id=ce9fe4f82f1f43e3be83457d040809b3)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:58.369127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972e413-44f4-456f-a7d2-def4f7e24c8d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:58.377105 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882443c-e3f3-4d6e-8449-10b685c1cfa4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:58.378224 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 196/387] 10.222.0.62 () {56 vars in 1180 bytes} [Fri Sep 9 11:52:58 2022] GET /identity/v3/groups/ce9fe4f82f1f43e3be83457d040809b3 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:58.442550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972e413-44f4-456f-a7d2-def4f7e24c8d None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:58.512537 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882443c-e3f3-4d6e-8449-10b685c1cfa4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:58.580577 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:58.595952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6fa585-554b-446e-b284-7aba945d5b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:58.596596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fa585-554b-446e-b284-7aba945d5b19 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:58.596928 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fa585-554b-446e-b284-7aba945d5b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:58.597202 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fa585-554b-446e-b284-7aba945d5b19 None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:58.598378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6fa585-554b-446e-b284-7aba945d5b19 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:58.599829 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6fa585-554b-446e-b284-7aba945d5b19 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:58.601455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 192/388] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:58 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:58.607643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6fa585-554b-446e-b284-7aba945d5b19 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:58.652878 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972e413-44f4-456f-a7d2-def4f7e24c8d None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:58.715326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:58.727273 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321b304-94f4-46bc-95fa-3c89cbe83314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:58.727937 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321b304-94f4-46bc-95fa-3c89cbe83314 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:58.728214 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321b304-94f4-46bc-95fa-3c89cbe83314 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:58.728482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321b304-94f4-46bc-95fa-3c89cbe83314 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:58.749749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321b304-94f4-46bc-95fa-3c89cbe83314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-1321b304-94f4-46bc-95fa-3c89cbe83314 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:58.758366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:58.759279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:58.760173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:58.760173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:58.760173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:58.760173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:58.760173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:58.760173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:58.760173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:58.760173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:58.760173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:58.760173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:58.760173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:58.760173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:52:58.765667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 197/389] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:52:58.771476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321b304-94f4-46bc-95fa-3c89cbe83314 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:58.799840 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6fa585-554b-446e-b284-7aba945d5b19 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:58.840802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26900c-0499-4c77-a2bb-c94fa37b757d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:58.841520 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26900c-0499-4c77-a2bb-c94fa37b757d None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:58.841914 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26900c-0499-4c77-a2bb-c94fa37b757d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:58.842213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26900c-0499-4c77-a2bb-c94fa37b757d None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa/groups/ce9fe4f82f1f43e3be83457d040809b3/roles/d43148a29472443ba7dfe2424ec60baf` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:58.849384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a26900c-0499-4c77-a2bb-c94fa37b757d None admin] RBAC: Authorizing `identity:create_grant(project_id=4f1f1e3af21d472cb21dc183a536cdfa, group_id=ce9fe4f82f1f43e3be83457d040809b3, role_id=d43148a29472443ba7dfe2424ec60baf)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:58.851091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a26900c-0499-4c77-a2bb-c94fa37b757d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:58.880859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 193/390] 10.222.0.62 () {58 vars in 1442 bytes} [Fri Sep 9 11:52:58 2022] PUT /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa/groups/ce9fe4f82f1f43e3be83457d040809b3/roles/d43148a29472443ba7dfe2424ec60baf => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:52:58.882242 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ddac6-d5d1-4959-b2fb-0c7e8fdf42e0 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:58.882526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ddac6-d5d1-4959-b2fb-0c7e8fdf42e0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:58.882795 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ddac6-d5d1-4959-b2fb-0c7e8fdf42e0 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:58.890768 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 194/391] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:58.896325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10c0fc-91ec-4419-8d6d-7d231a0df59c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:58.896608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10c0fc-91ec-4419-8d6d-7d231a0df59c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:58.896879 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10c0fc-91ec-4419-8d6d-7d231a0df59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:58.939279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-da10c0fc-91ec-4419-8d6d-7d231a0df59c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:58.940545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-da10c0fc-91ec-4419-8d6d-7d231a0df59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:58.960801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321b304-94f4-46bc-95fa-3c89cbe83314 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:59.009216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97660858-9709-4c9a-8990-8f127930e7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:59.009829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97660858-9709-4c9a-8990-8f127930e7bf None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:59.010102 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97660858-9709-4c9a-8990-8f127930e7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:59.010394 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97660858-9709-4c9a-8990-8f127930e7bf None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:59.011137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97660858-9709-4c9a-8990-8f127930e7bf None admin] RBAC: Adding query filter params (name=member) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:59.011441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97660858-9709-4c9a-8990-8f127930e7bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:59.012887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97660858-9709-4c9a-8990-8f127930e7bf None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:59.026451 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:59.032878 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 198/392] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:59.038688 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97660858-9709-4c9a-8990-8f127930e7bf None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:59.043795 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 195/393] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:52:59.097358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:59.163310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97660858-9709-4c9a-8990-8f127930e7bf None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:59.205457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee7cc5-2002-4fc8-a4bf-b232bbe56d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:59.206105 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee7cc5-2002-4fc8-a4bf-b232bbe56d6b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:59.206407 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee7cc5-2002-4fc8-a4bf-b232bbe56d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:59.206675 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee7cc5-2002-4fc8-a4bf-b232bbe56d6b None admin] PATH_INFO: `/v3/groups/ce9fe4f82f1f43e3be83457d040809b3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:59.207796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee7cc5-2002-4fc8-a4bf-b232bbe56d6b None admin] RBAC: Authorizing `identity:get_group(group_id=ce9fe4f82f1f43e3be83457d040809b3)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:59.209234 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee7cc5-2002-4fc8-a4bf-b232bbe56d6b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:59.211724 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 199/394] 10.222.0.62 () {56 vars in 1180 bytes} [Fri Sep 9 11:52:59 2022] GET /identity/v3/groups/ce9fe4f82f1f43e3be83457d040809b3 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:59.217219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10c0fc-91ec-4419-8d6d-7d231a0df59c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:59.337327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10c0fc-91ec-4419-8d6d-7d231a0df59c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:59.373839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50b7de-3eb5-4ec3-b93e-fa081887c14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:59.374414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50b7de-3eb5-4ec3-b93e-fa081887c14a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:59.374689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50b7de-3eb5-4ec3-b93e-fa081887c14a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:59.374953 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50b7de-3eb5-4ec3-b93e-fa081887c14a None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:59.376112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc50b7de-3eb5-4ec3-b93e-fa081887c14a None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:59.377525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc50b7de-3eb5-4ec3-b93e-fa081887c14a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:59.379484 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 196/395] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:59 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:59.385850 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee7cc5-2002-4fc8-a4bf-b232bbe56d6b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:59.417393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50b7de-3eb5-4ec3-b93e-fa081887c14a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:59.480669 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:59.532985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50b7de-3eb5-4ec3-b93e-fa081887c14a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:59.563323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee7cc5-2002-4fc8-a4bf-b232bbe56d6b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:59.573713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e2f14f-3b84-4db7-901c-dc392ccb6ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:59.574396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e2f14f-3b84-4db7-901c-dc392ccb6ecc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:59.574675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e2f14f-3b84-4db7-901c-dc392ccb6ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:59.574948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e2f14f-3b84-4db7-901c-dc392ccb6ecc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:59.586560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e2f14f-3b84-4db7-901c-dc392ccb6ecc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-40e2f14f-3b84-4db7-901c-dc392ccb6ecc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:52:59.593007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:52:59.593916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:52:59.594781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:52:59.594781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:52:59.594781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:52:59.594781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:52:59.594781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:52:59.594781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:52:59.594781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:52:59.594781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:52:59.594781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:52:59.594781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:52:59.594781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:52:59.594781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:52:59.594781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 197/396] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:52:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:52:59.600754 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e2f14f-3b84-4db7-901c-dc392ccb6ecc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:59.608997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:59.636931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9fe80-e39f-49ee-8c28-bf95bd41369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:59.637517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9fe80-e39f-49ee-8c28-bf95bd41369a None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:59.637793 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9fe80-e39f-49ee-8c28-bf95bd41369a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:59.638065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9fe80-e39f-49ee-8c28-bf95bd41369a None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa/groups/ce9fe4f82f1f43e3be83457d040809b3/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:59.641544 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9fe80-e39f-49ee-8c28-bf95bd41369a None admin] RBAC: Authorizing `identity:create_grant(project_id=4f1f1e3af21d472cb21dc183a536cdfa, group_id=ce9fe4f82f1f43e3be83457d040809b3, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:59.643040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9fe80-e39f-49ee-8c28-bf95bd41369a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:59.666353 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 200/397] 10.222.0.62 () {58 vars in 1442 bytes} [Fri Sep 9 11:52:59 2022] PUT /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa/groups/ce9fe4f82f1f43e3be83457d040809b3/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:52:59.759448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e2f14f-3b84-4db7-901c-dc392ccb6ecc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:59.779453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:52:59.800434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488507d-f135-4c60-a791-bd9b87531509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:59.801115 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488507d-f135-4c60-a791-bd9b87531509 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:59.801326 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488507d-f135-4c60-a791-bd9b87531509 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:59.801601 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488507d-f135-4c60-a791-bd9b87531509 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:59.802308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488507d-f135-4c60-a791-bd9b87531509 None admin] RBAC: Adding query filter params (name=member) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:52:59.802612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488507d-f135-4c60-a791-bd9b87531509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:59.804021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488507d-f135-4c60-a791-bd9b87531509 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:59.814527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 198/398] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:52:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:52:59.820506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9fe80-e39f-49ee-8c28-bf95bd41369a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:52:59.929367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9fe80-e39f-49ee-8c28-bf95bd41369a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:52:59.942401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e695-fd3f-475c-ace3-c038f6000d39 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:59.943373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e695-fd3f-475c-ace3-c038f6000d39 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:59.943873 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e695-fd3f-475c-ace3-c038f6000d39 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:59.946600 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 199/399] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:52:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:52:59.956026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16826382-64a2-4c39-96ea-99780f116b7d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:59.957036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16826382-64a2-4c39-96ea-99780f116b7d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:59.957519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16826382-64a2-4c39-96ea-99780f116b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:59.975334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e6766-8176-49d7-8a1f-d16d6b26e7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:52:59.976590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e6766-8176-49d7-8a1f-d16d6b26e7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:52:59.977007 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e6766-8176-49d7-8a1f-d16d6b26e7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:52:59.977413 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e6766-8176-49d7-8a1f-d16d6b26e7b3 None admin] PATH_INFO: `/v3/groups/ce9fe4f82f1f43e3be83457d040809b3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:52:59.978708 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e6766-8176-49d7-8a1f-d16d6b26e7b3 None admin] RBAC: Authorizing `identity:get_group(group_id=ce9fe4f82f1f43e3be83457d040809b3)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:52:59.980300 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e6766-8176-49d7-8a1f-d16d6b26e7b3 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:52:59.981401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-16826382-64a2-4c39-96ea-99780f116b7d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:52:59.982025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 201/400] 10.222.0.62 () {56 vars in 1180 bytes} [Fri Sep 9 11:52:59 2022] GET /identity/v3/groups/ce9fe4f82f1f43e3be83457d040809b3 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:52:59.983200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-16826382-64a2-4c39-96ea-99780f116b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:52:59.987673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e6766-8176-49d7-8a1f-d16d6b26e7b3 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:00.067361 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 200/401] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:53:00.095237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:00.205869 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e6766-8176-49d7-8a1f-d16d6b26e7b3 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:00.260253 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b9578-8b2d-4291-b7a6-bff69c0238f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:00.260897 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b9578-8b2d-4291-b7a6-bff69c0238f3 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:00.261177 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b9578-8b2d-4291-b7a6-bff69c0238f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:00.261449 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b9578-8b2d-4291-b7a6-bff69c0238f3 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:00.264585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b9578-8b2d-4291-b7a6-bff69c0238f3 None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:00.266047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b9578-8b2d-4291-b7a6-bff69c0238f3 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:00.267746 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 202/402] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:52:59 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:00.269706 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315c3f7-3a89-4f5d-a85b-1f51738b36a4 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:00.269991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315c3f7-3a89-4f5d-a85b-1f51738b36a4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:00.270300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315c3f7-3a89-4f5d-a85b-1f51738b36a4 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:00.273833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b9578-8b2d-4291-b7a6-bff69c0238f3 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:00.274245 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 201/403] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:00.281488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949eb30-3b57-4e8c-8913-e17181d76dad None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:00.281770 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949eb30-3b57-4e8c-8913-e17181d76dad None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:00.282045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949eb30-3b57-4e8c-8913-e17181d76dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:00.302459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7949eb30-3b57-4e8c-8913-e17181d76dad None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:00.303646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7949eb30-3b57-4e8c-8913-e17181d76dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:00.404252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 202/404] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:00.415578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949eb30-3b57-4e8c-8913-e17181d76dad None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:00.461454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b9578-8b2d-4291-b7a6-bff69c0238f3 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:00.477923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:00.493853 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:00.519469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c42d9-df32-4375-a2ed-7904e8bfbd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:00.520751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c42d9-df32-4375-a2ed-7904e8bfbd68 None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:00.522371 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c42d9-df32-4375-a2ed-7904e8bfbd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:00.523552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c42d9-df32-4375-a2ed-7904e8bfbd68 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71/groups/ce9fe4f82f1f43e3be83457d040809b3/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:00.528575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c42d9-df32-4375-a2ed-7904e8bfbd68 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a96e4f9e7134925af6e684f79748f71, group_id=ce9fe4f82f1f43e3be83457d040809b3, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:00.530283 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c42d9-df32-4375-a2ed-7904e8bfbd68 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:00.552754 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 203/405] 10.222.0.62 () {58 vars in 1442 bytes} [Fri Sep 9 11:53:00 2022] PUT /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71/groups/ce9fe4f82f1f43e3be83457d040809b3/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:00.581496 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949eb30-3b57-4e8c-8913-e17181d76dad None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:00.631997 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cddf16-a266-42fb-9a80-c5499db2b8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:00.633674 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cddf16-a266-42fb-9a80-c5499db2b8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:00.634218 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cddf16-a266-42fb-9a80-c5499db2b8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:00.634773 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cddf16-a266-42fb-9a80-c5499db2b8f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:00.651303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cddf16-a266-42fb-9a80-c5499db2b8f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-f8cddf16-a266-42fb-9a80-c5499db2b8f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:00.657147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:00.658151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:00.658993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:00.658993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:00.658993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:00.658993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:00.658993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:00.658993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:00.658993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:00.658993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:00.658993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:00.658993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:00.658993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:53:00.658993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:00.661495 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 203/406] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:53:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:00.667869 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c42d9-df32-4375-a2ed-7904e8bfbd68 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:00.708968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cddf16-a266-42fb-9a80-c5499db2b8f7 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:00.823281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c42d9-df32-4375-a2ed-7904e8bfbd68 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:00.852715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:00.856274 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cddf16-a266-42fb-9a80-c5499db2b8f7 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:00.872487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8e2c8-f5a4-484a-93bd-389d8e28df18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:00.873489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8e2c8-f5a4-484a-93bd-389d8e28df18 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:00.873965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8e2c8-f5a4-484a-93bd-389d8e28df18 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:00.875145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8e2c8-f5a4-484a-93bd-389d8e28df18 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:00.876121 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8e2c8-f5a4-484a-93bd-389d8e28df18 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:00.876737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8e2c8-f5a4-484a-93bd-389d8e28df18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:00.878388 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8e2c8-f5a4-484a-93bd-389d8e28df18 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:00.888530 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 204/407] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:00.894690 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8e2c8-f5a4-484a-93bd-389d8e28df18 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:00.896512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debcfb9-29be-4ecc-bc86-5fc21f0297b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:00.897155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debcfb9-29be-4ecc-bc86-5fc21f0297b2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:00.897428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debcfb9-29be-4ecc-bc86-5fc21f0297b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:00.897694 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debcfb9-29be-4ecc-bc86-5fc21f0297b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:00.909299 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debcfb9-29be-4ecc-bc86-5fc21f0297b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-3debcfb9-29be-4ecc-bc86-5fc21f0297b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:00.917881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:00.918954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:00.919763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:00.919763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:00.919763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:00.919763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:00.919763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:00.919763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:00.919763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:00.919763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:00.919763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:00.919763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:00.919763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:00.919763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:00.919763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 204/408] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:53:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:00.921948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debcfb9-29be-4ecc-bc86-5fc21f0297b2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:01.002804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:01.067429 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debcfb9-29be-4ecc-bc86-5fc21f0297b2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:01.118043 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8e2c8-f5a4-484a-93bd-389d8e28df18 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:01.130757 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69502495-0f70-4d67-910c-85d082ef728b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:01.131373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69502495-0f70-4d67-910c-85d082ef728b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:01.131653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69502495-0f70-4d67-910c-85d082ef728b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:01.131922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69502495-0f70-4d67-910c-85d082ef728b None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:01.132588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69502495-0f70-4d67-910c-85d082ef728b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:01.133418 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69502495-0f70-4d67-910c-85d082ef728b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:01.134797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69502495-0f70-4d67-910c-85d082ef728b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:01.155605 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 205/409] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:01.156493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69502495-0f70-4d67-910c-85d082ef728b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:01.184933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f3faf-e056-4526-972b-3a8936c627ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:01.185563 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f3faf-e056-4526-972b-3a8936c627ad None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:01.185837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f3faf-e056-4526-972b-3a8936c627ad None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:01.186104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f3faf-e056-4526-972b-3a8936c627ad None admin] PATH_INFO: `/v3/groups/ce9fe4f82f1f43e3be83457d040809b3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:01.187942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f3faf-e056-4526-972b-3a8936c627ad None admin] RBAC: Authorizing `identity:get_group(group_id=ce9fe4f82f1f43e3be83457d040809b3)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:01.189367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f3faf-e056-4526-972b-3a8936c627ad None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:01.192780 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 205/410] 10.222.0.62 () {56 vars in 1180 bytes} [Fri Sep 9 11:53:00 2022] GET /identity/v3/groups/ce9fe4f82f1f43e3be83457d040809b3 => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:01.197901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f3faf-e056-4526-972b-3a8936c627ad None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:01.200350 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:01.328631 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69502495-0f70-4d67-910c-85d082ef728b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:01.363969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d109a3-c22b-4a6c-9954-693aad933ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:01.364941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d109a3-c22b-4a6c-9954-693aad933ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:01.365316 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d109a3-c22b-4a6c-9954-693aad933ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:01.365680 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d109a3-c22b-4a6c-9954-693aad933ad9 None admin] PATH_INFO: `/v3/groups/5deb34b4087e46c4a4cbe16c7fe15338` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:01.366950 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d109a3-c22b-4a6c-9954-693aad933ad9 None admin] RBAC: Authorizing `identity:get_group(group_id=5deb34b4087e46c4a4cbe16c7fe15338)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:01.369037 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d109a3-c22b-4a6c-9954-693aad933ad9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:01.371039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 206/411] 10.222.0.62 () {56 vars in 1180 bytes} [Fri Sep 9 11:53:01 2022] GET /identity/v3/groups/5deb34b4087e46c4a4cbe16c7fe15338 => generated 229 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:01.383492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d109a3-c22b-4a6c-9954-693aad933ad9 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:01.387977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f3faf-e056-4526-972b-3a8936c627ad None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:01.395454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:01.426568 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93fe558-6786-46f5-849d-c7380e585fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:01.427212 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fe558-6786-46f5-849d-c7380e585fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:01.427514 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fe558-6786-46f5-849d-c7380e585fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:01.427770 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fe558-6786-46f5-849d-c7380e585fe4 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:01.428943 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93fe558-6786-46f5-849d-c7380e585fe4 None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:01.430929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93fe558-6786-46f5-849d-c7380e585fe4 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:01.433148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 206/412] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:53:01 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:01.438432 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93fe558-6786-46f5-849d-c7380e585fe4 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:01.553199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93fe558-6786-46f5-849d-c7380e585fe4 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:01.593812 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065a2c0-bdc1-44b9-a0b9-dabc3a8c715f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:01.594526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065a2c0-bdc1-44b9-a0b9-dabc3a8c715f None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:01.594842 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065a2c0-bdc1-44b9-a0b9-dabc3a8c715f None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:01.595111 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065a2c0-bdc1-44b9-a0b9-dabc3a8c715f None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71/groups/ce9fe4f82f1f43e3be83457d040809b3/roles/d43148a29472443ba7dfe2424ec60baf` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:01.598651 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065a2c0-bdc1-44b9-a0b9-dabc3a8c715f None admin] RBAC: Authorizing `identity:create_grant(project_id=3a96e4f9e7134925af6e684f79748f71, group_id=ce9fe4f82f1f43e3be83457d040809b3, role_id=d43148a29472443ba7dfe2424ec60baf)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:01.600291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065a2c0-bdc1-44b9-a0b9-dabc3a8c715f None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:01.628746 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 207/413] 10.222.0.62 () {58 vars in 1442 bytes} [Fri Sep 9 11:53:01 2022] PUT /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71/groups/ce9fe4f82f1f43e3be83457d040809b3/roles/d43148a29472443ba7dfe2424ec60baf => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:53:01.637390 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:01.658681 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d109a3-c22b-4a6c-9954-693aad933ad9 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:01.708978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312c61c-4367-4f49-894a-f58f3641f35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:01.710013 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312c61c-4367-4f49-894a-f58f3641f35b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:01.710450 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312c61c-4367-4f49-894a-f58f3641f35b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:01.710913 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312c61c-4367-4f49-894a-f58f3641f35b None admin] PATH_INFO: `/v3/projects/fe7d1fa6c42548518e72832336a573e5` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:01.712312 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312c61c-4367-4f49-894a-f58f3641f35b None admin] RBAC: Authorizing `identity:get_project(project_id=fe7d1fa6c42548518e72832336a573e5)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:01.713893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312c61c-4367-4f49-894a-f58f3641f35b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:01.716122 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 207/414] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:53:01 2022] GET /identity/v3/projects/fe7d1fa6c42548518e72832336a573e5 => generated 344 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:01.722148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065a2c0-bdc1-44b9-a0b9-dabc3a8c715f None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:01.774001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:01.882701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065a2c0-bdc1-44b9-a0b9-dabc3a8c715f None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:01.921623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9548a-36dc-4310-913c-366c853f848d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:01.922598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9548a-36dc-4310-913c-366c853f848d None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:01.923013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9548a-36dc-4310-913c-366c853f848d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:01.923448 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9548a-36dc-4310-913c-366c853f848d None admin] PATH_INFO: `/v3/projects/fe7d1fa6c42548518e72832336a573e5/groups/5deb34b4087e46c4a4cbe16c7fe15338/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:01.925493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9548a-36dc-4310-913c-366c853f848d None admin] RBAC: Authorizing `identity:create_grant(project_id=fe7d1fa6c42548518e72832336a573e5, group_id=5deb34b4087e46c4a4cbe16c7fe15338, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:01.927081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9548a-36dc-4310-913c-366c853f848d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:01.944532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 208/415] 10.222.0.62 () {58 vars in 1442 bytes} [Fri Sep 9 11:53:01 2022] PUT /identity/v3/projects/fe7d1fa6c42548518e72832336a573e5/groups/5deb34b4087e46c4a4cbe16c7fe15338/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:01.953818 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb1e27-d444-4d4e-a5a7-55427fea54d4 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:01.954434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb1e27-d444-4d4e-a5a7-55427fea54d4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:01.954434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb1e27-d444-4d4e-a5a7-55427fea54d4 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:01.956166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 208/416] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:01.961933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86fa60-96ea-43d8-93cd-b37290c8b50e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:01.962316 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86fa60-96ea-43d8-93cd-b37290c8b50e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:01.962598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86fa60-96ea-43d8-93cd-b37290c8b50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:01.983880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-3d86fa60-96ea-43d8-93cd-b37290c8b50e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:01.985559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-3d86fa60-96ea-43d8-93cd-b37290c8b50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:02.064666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 209/417] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:02.286328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26528d51-7789-4d11-98e2-3f4aa8dbc77a None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:02.286664 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26528d51-7789-4d11-98e2-3f4aa8dbc77a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:02.286871 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26528d51-7789-4d11-98e2-3f4aa8dbc77a None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:02.288498 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 209/418] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:02.293858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60199769-1353-444a-ae8c-56b44c69d139 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:02.294154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60199769-1353-444a-ae8c-56b44c69d139 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:02.294422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60199769-1353-444a-ae8c-56b44c69d139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:02.318086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-60199769-1353-444a-ae8c-56b44c69d139 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:02.319299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-60199769-1353-444a-ae8c-56b44c69d139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:02.346724 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:02.411384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 210/419] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:02.485528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26528d51-7789-4d11-98e2-3f4aa8dbc77a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:02.614817 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26528d51-7789-4d11-98e2-3f4aa8dbc77a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:02.657495 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b76707-85c1-498e-aadb-6898dfd884a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:02.658160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b76707-85c1-498e-aadb-6898dfd884a1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:02.658445 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b76707-85c1-498e-aadb-6898dfd884a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:02.658718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b76707-85c1-498e-aadb-6898dfd884a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:02.679369 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b76707-85c1-498e-aadb-6898dfd884a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-83b76707-85c1-498e-aadb-6898dfd884a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:02.685042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:02.685930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:02.686766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:02.686766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:02.686766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:02.686766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:02.686766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:02.686766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:02.686766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:02.686766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:02.686766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:02.686766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:02.686766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:53:02.686766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:02.686766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 210/420] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:53:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:02.693478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60199769-1353-444a-ae8c-56b44c69d139 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:02.735745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b76707-85c1-498e-aadb-6898dfd884a1 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:02.766038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:02.862225 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60199769-1353-444a-ae8c-56b44c69d139 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:02.935975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89dbcc-3b43-4930-9a64-a4018eea93b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:02.936643 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89dbcc-3b43-4930-9a64-a4018eea93b9 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:02.936956 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89dbcc-3b43-4930-9a64-a4018eea93b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:02.937245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89dbcc-3b43-4930-9a64-a4018eea93b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:02.941739 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89dbcc-3b43-4930-9a64-a4018eea93b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:02.942080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89dbcc-3b43-4930-9a64-a4018eea93b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:02.943693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89dbcc-3b43-4930-9a64-a4018eea93b9 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:02.947376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b76707-85c1-498e-aadb-6898dfd884a1 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:02.964460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 211/421] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:02.970624 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89dbcc-3b43-4930-9a64-a4018eea93b9 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:03.007325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:03.024259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e13d6-497f-4e49-bdcd-ef883bf4fb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:03.024967 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e13d6-497f-4e49-bdcd-ef883bf4fb11 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:03.025297 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e13d6-497f-4e49-bdcd-ef883bf4fb11 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:03.025574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e13d6-497f-4e49-bdcd-ef883bf4fb11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:03.053292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e13d6-497f-4e49-bdcd-ef883bf4fb11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-5e8e13d6-497f-4e49-bdcd-ef883bf4fb11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:03.058333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:03.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:03.060012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:03.060012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:03.060012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:03.060012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:03.060012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:03.060012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:03.060012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:03.060012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:03.060012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:03.060012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:03.060012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:53:03.060012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:03.065423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 211/422] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:03.069186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e13d6-497f-4e49-bdcd-ef883bf4fb11 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:03.174441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89dbcc-3b43-4930-9a64-a4018eea93b9 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:03.228676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9160db-bea1-4eb6-8dee-95672e0da79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:03.229298 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9160db-bea1-4eb6-8dee-95672e0da79a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:03.229577 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9160db-bea1-4eb6-8dee-95672e0da79a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:03.229845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9160db-bea1-4eb6-8dee-95672e0da79a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:03.233373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:03.305618 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e13d6-497f-4e49-bdcd-ef883bf4fb11 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:03.315244 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9160db-bea1-4eb6-8dee-95672e0da79a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:03.316730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9160db-bea1-4eb6-8dee-95672e0da79a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:03.343332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:03.389149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bdcd3-3c43-41c7-b230-b980ddfb7ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:03.389771 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bdcd3-3c43-41c7-b230-b980ddfb7ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:03.390050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bdcd3-3c43-41c7-b230-b980ddfb7ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:03.390345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bdcd3-3c43-41c7-b230-b980ddfb7ed8 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:03.391022 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9bdcd3-3c43-41c7-b230-b980ddfb7ed8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:03.391323 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9bdcd3-3c43-41c7-b230-b980ddfb7ed8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:03.392690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9bdcd3-3c43-41c7-b230-b980ddfb7ed8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:03.417850 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 212/423] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:53:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:03.420054 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bdcd3-3c43-41c7-b230-b980ddfb7ed8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:03.469612 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 212/424] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:53:03.471003 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62516015-b20e-4780-817a-468757339138 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:03.471289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62516015-b20e-4780-817a-468757339138 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:03.471560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62516015-b20e-4780-817a-468757339138 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:03.473442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 213/425] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:03.478960 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62516015-b20e-4780-817a-468757339138 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:03.545757 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bdcd3-3c43-41c7-b230-b980ddfb7ed8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:03.589511 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87a858-3ecc-471b-b7c8-fe670626f868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:03.590138 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87a858-3ecc-471b-b7c8-fe670626f868 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:03.590477 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87a858-3ecc-471b-b7c8-fe670626f868 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:03.590799 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87a858-3ecc-471b-b7c8-fe670626f868 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:03.600974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:03.610714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62516015-b20e-4780-817a-468757339138 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:03.632965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:03.658953 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087df6b-ab5b-4a68-b2be-9a78ef5cd516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:03.660143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087df6b-ab5b-4a68-b2be-9a78ef5cd516 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:03.660600 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087df6b-ab5b-4a68-b2be-9a78ef5cd516 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:03.661040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087df6b-ab5b-4a68-b2be-9a78ef5cd516 None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:03.663172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087df6b-ab5b-4a68-b2be-9a78ef5cd516 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:03.666267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087df6b-ab5b-4a68-b2be-9a78ef5cd516 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:03.667969 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 214/426] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:53:03 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:03.670225 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2782d-e412-48b4-a350-c4f7a3b94242 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:03.670510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2782d-e412-48b4-a350-c4f7a3b94242 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:03.670785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2782d-e412-48b4-a350-c4f7a3b94242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:03.680640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87a858-3ecc-471b-b7c8-fe670626f868 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:03.682575 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87a858-3ecc-471b-b7c8-fe670626f868 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:03.696742 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-cdc2782d-e412-48b4-a350-c4f7a3b94242 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:03.698280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-cdc2782d-e412-48b4-a350-c4f7a3b94242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:03.786382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 215/427] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:03.787641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2782d-e412-48b4-a350-c4f7a3b94242 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:03.800300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 213/428] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:53:03.805378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87a858-3ecc-471b-b7c8-fe670626f868 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:03.851985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:03.964292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87a858-3ecc-471b-b7c8-fe670626f868 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:03.970220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2782d-e412-48b4-a350-c4f7a3b94242 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:03.989879 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:04.024024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e8d83-2004-475b-b859-c98a6f68aa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:04.024865 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508065df-763a-447b-b7f7-b47d861bb446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:04.026005 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508065df-763a-447b-b7f7-b47d861bb446 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:04.026440 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508065df-763a-447b-b7f7-b47d861bb446 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:04.027295 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e8d83-2004-475b-b859-c98a6f68aa12 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:04.027709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508065df-763a-447b-b7f7-b47d861bb446 None admin] PATH_INFO: `/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:04.028627 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e8d83-2004-475b-b859-c98a6f68aa12 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:04.029033 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e8d83-2004-475b-b859-c98a6f68aa12 None admin] PATH_INFO: `/v3/groups/ce9fe4f82f1f43e3be83457d040809b3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:04.029773 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508065df-763a-447b-b7f7-b47d861bb446 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1f1e3af21d472cb21dc183a536cdfa)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:04.031893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508065df-763a-447b-b7f7-b47d861bb446 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:04.032631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e8d83-2004-475b-b859-c98a6f68aa12 None admin] RBAC: Authorizing `identity:get_group(group_id=ce9fe4f82f1f43e3be83457d040809b3)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:04.034234 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e8d83-2004-475b-b859-c98a6f68aa12 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:04.038473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 214/429] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:53:03 2022] GET /identity/v3/projects/4f1f1e3af21d472cb21dc183a536cdfa => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:04.038863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 216/430] 10.222.0.62 () {56 vars in 1180 bytes} [Fri Sep 9 11:53:03 2022] GET /identity/v3/groups/ce9fe4f82f1f43e3be83457d040809b3 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:04.049297 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508065df-763a-447b-b7f7-b47d861bb446 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:04.050601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e8d83-2004-475b-b859-c98a6f68aa12 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:04.149783 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508065df-763a-447b-b7f7-b47d861bb446 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:04.157856 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:04.189901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824d52e-faac-418d-967f-393ed8d44af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:04.190544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824d52e-faac-418d-967f-393ed8d44af8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:04.190816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824d52e-faac-418d-967f-393ed8d44af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:04.191080 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824d52e-faac-418d-967f-393ed8d44af8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:04.191807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824d52e-faac-418d-967f-393ed8d44af8 None admin] RBAC: Adding query filter params (group.id=ce9fe4f82f1f43e3be83457d040809b3, role.id=d43148a29472443ba7dfe2424ec60baf, scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:04.192111 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824d52e-faac-418d-967f-393ed8d44af8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:04.193543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824d52e-faac-418d-967f-393ed8d44af8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:04.205882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 215/431] 10.222.0.62 () {56 vars in 1376 bytes} [Fri Sep 9 11:53:04 2022] GET /identity/v3/role_assignments?group.id=ce9fe4f82f1f43e3be83457d040809b3&scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa&role.id=d43148a29472443ba7dfe2424ec60baf => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:04.208200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824d52e-faac-418d-967f-393ed8d44af8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:04.284083 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e8d83-2004-475b-b859-c98a6f68aa12 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:04.324033 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25baba9c-26d8-4ab6-a746-68e870423884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:04.324680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25baba9c-26d8-4ab6-a746-68e870423884 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:04.324952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25baba9c-26d8-4ab6-a746-68e870423884 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:04.325214 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25baba9c-26d8-4ab6-a746-68e870423884 None admin] PATH_INFO: `/v3/groups/ce9fe4f82f1f43e3be83457d040809b3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:04.326463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25baba9c-26d8-4ab6-a746-68e870423884 None admin] RBAC: Authorizing `identity:get_group(group_id=ce9fe4f82f1f43e3be83457d040809b3)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:04.327942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25baba9c-26d8-4ab6-a746-68e870423884 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:04.329900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 217/432] 10.222.0.62 () {56 vars in 1180 bytes} [Fri Sep 9 11:53:04 2022] GET /identity/v3/groups/ce9fe4f82f1f43e3be83457d040809b3 => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:04.331182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f44bd-dbab-4ec3-a1ad-8e9ea476b341 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:04.331403 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f44bd-dbab-4ec3-a1ad-8e9ea476b341 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:04.331668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f44bd-dbab-4ec3-a1ad-8e9ea476b341 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:04.338471 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 218/433] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:04.339739 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f44bd-dbab-4ec3-a1ad-8e9ea476b341 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:04.355618 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824d52e-faac-418d-967f-393ed8d44af8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:04.397587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:04.404892 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95a300-c23f-4ec9-b4e5-380311ef2138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:04.405859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95a300-c23f-4ec9-b4e5-380311ef2138 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:04.406425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95a300-c23f-4ec9-b4e5-380311ef2138 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:04.406856 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95a300-c23f-4ec9-b4e5-380311ef2138 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:04.419419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95a300-c23f-4ec9-b4e5-380311ef2138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-cd95a300-c23f-4ec9-b4e5-380311ef2138 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:04.425271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:04.426415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:04.427239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:04.427239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:04.427239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:04.427239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:04.427239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:04.427239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:04.427239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:04.427239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:04.427239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:04.427239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:04.427239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:53:04.427239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:04.433625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 216/434] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:04.433625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a5dc1-91a7-46f4-9efb-ec7a11a9f5e1 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:04.433625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a5dc1-91a7-46f4-9efb-ec7a11a9f5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:04.433625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a5dc1-91a7-46f4-9efb-ec7a11a9f5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:04.452998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b64a5dc1-91a7-46f4-9efb-ec7a11a9f5e1 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:04.454522 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b64a5dc1-91a7-46f4-9efb-ec7a11a9f5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:04.457202 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f44bd-dbab-4ec3-a1ad-8e9ea476b341 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:04.492708 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:04.511510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d57cd-7db5-43af-a930-d9fea988cf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:04.512126 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d57cd-7db5-43af-a930-d9fea988cf49 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:04.512399 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d57cd-7db5-43af-a930-d9fea988cf49 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:04.512665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d57cd-7db5-43af-a930-d9fea988cf49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:04.514788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d57cd-7db5-43af-a930-d9fea988cf49 None admin] RBAC: Adding query filter params (group.id=ce9fe4f82f1f43e3be83457d040809b3, role.id=e385f7d11fac4932a910ec0ef95e2a43, scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:04.514788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d57cd-7db5-43af-a930-d9fea988cf49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:04.515259 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d57cd-7db5-43af-a930-d9fea988cf49 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:04.521494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 219/435] 10.222.0.62 () {56 vars in 1376 bytes} [Fri Sep 9 11:53:04 2022] GET /identity/v3/role_assignments?group.id=ce9fe4f82f1f43e3be83457d040809b3&scope.project.id=4f1f1e3af21d472cb21dc183a536cdfa&role.id=e385f7d11fac4932a910ec0ef95e2a43 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:04.522308 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d57cd-7db5-43af-a930-d9fea988cf49 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:04.573745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 217/436] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:04.604435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a605e-6107-4174-a2d8-341267920947 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:04.604755 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a605e-6107-4174-a2d8-341267920947 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:04.604985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a605e-6107-4174-a2d8-341267920947 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:04.606747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 218/437] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:53:04.612585 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40873d-f7b5-4efb-9640-f6bf6e313b39 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:04.612857 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40873d-f7b5-4efb-9640-f6bf6e313b39 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:04.613130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40873d-f7b5-4efb-9640-f6bf6e313b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:04.633277 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ac40873d-f7b5-4efb-9640-f6bf6e313b39 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:04.634544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ac40873d-f7b5-4efb-9640-f6bf6e313b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:04.642265 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d57cd-7db5-43af-a930-d9fea988cf49 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:04.658969 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:04.682450 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42a2ef-4947-43f4-854c-8cc21b33deab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:04.683162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42a2ef-4947-43f4-854c-8cc21b33deab None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:04.683511 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42a2ef-4947-43f4-854c-8cc21b33deab None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:04.683851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42a2ef-4947-43f4-854c-8cc21b33deab None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:04.684615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42a2ef-4947-43f4-854c-8cc21b33deab None admin] RBAC: Adding query filter params (name=member) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:04.684989 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42a2ef-4947-43f4-854c-8cc21b33deab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:04.686624 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42a2ef-4947-43f4-854c-8cc21b33deab None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:04.701630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 220/438] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:53:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:04.703298 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42a2ef-4947-43f4-854c-8cc21b33deab None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:04.730303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 219/439] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:53:04.829262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42a2ef-4947-43f4-854c-8cc21b33deab None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:04.864626 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e720fd9-d39d-4041-ad9d-584787a3492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:04.865233 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e720fd9-d39d-4041-ad9d-584787a3492a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:04.865504 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e720fd9-d39d-4041-ad9d-584787a3492a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:04.865765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e720fd9-d39d-4041-ad9d-584787a3492a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:04.894561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40873d-f7b5-4efb-9640-f6bf6e313b39 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:04.916374 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:04.958869 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e720fd9-d39d-4041-ad9d-584787a3492a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:04.960531 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e720fd9-d39d-4041-ad9d-584787a3492a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:05.019746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:05.031341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40873d-f7b5-4efb-9640-f6bf6e313b39 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:05.073316 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d9df1-9702-45a2-aa94-9a958c211d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:05.073957 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d9df1-9702-45a2-aa94-9a958c211d0f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:05.074325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d9df1-9702-45a2-aa94-9a958c211d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:05.074535 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d9df1-9702-45a2-aa94-9a958c211d0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:05.082038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 221/440] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:53:05.083155 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e720fd9-d39d-4041-ad9d-584787a3492a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:05.090990 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d9df1-9702-45a2-aa94-9a958c211d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-7f3d9df1-9702-45a2-aa94-9a958c211d0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:05.095926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:05.097016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:05.097810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:05.097810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:05.097810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:05.097810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:05.097810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:05.097810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:05.097810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:05.097810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:05.097810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:05.097810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:05.097810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:53:05.097810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:05.097810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 220/441] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:53:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:05.099201 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d9df1-9702-45a2-aa94-9a958c211d0f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:05.172805 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:05.202488 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e720fd9-d39d-4041-ad9d-584787a3492a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:05.222849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d9df1-9702-45a2-aa94-9a958c211d0f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:05.239441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4a651-3611-4971-a7a8-3bf47685b295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:05.240071 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4a651-3611-4971-a7a8-3bf47685b295 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:05.240349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4a651-3611-4971-a7a8-3bf47685b295 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:05.240621 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4a651-3611-4971-a7a8-3bf47685b295 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:05.251571 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4a651-3611-4971-a7a8-3bf47685b295 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-c6f4a651-3611-4971-a7a8-3bf47685b295 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:05.259948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:05.260893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:05.261681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:05.261681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:05.261681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:05.261681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:05.261681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:05.261681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:05.261681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:05.261681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:05.261681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:05.261681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:05.261681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:05.261681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:05.263173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 222/442] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:53:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:05.263990 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a6c25-d6fe-4754-805c-55fbdca16536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:05.264230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4a651-3611-4971-a7a8-3bf47685b295 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:05.264750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a6c25-d6fe-4754-805c-55fbdca16536 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:05.264897 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a6c25-d6fe-4754-805c-55fbdca16536 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:05.265286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a6c25-d6fe-4754-805c-55fbdca16536 None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:05.266461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a6c25-d6fe-4754-805c-55fbdca16536 None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:05.267944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a6c25-d6fe-4754-805c-55fbdca16536 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:05.272229 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 221/443] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:53:05 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:05.272908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a6c25-d6fe-4754-805c-55fbdca16536 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:05.294973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:05.367990 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4a651-3611-4971-a7a8-3bf47685b295 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:05.375113 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a6c25-d6fe-4754-805c-55fbdca16536 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:05.401848 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacc644-fe2a-438d-9a7c-2646dab892aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:05.402473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacc644-fe2a-438d-9a7c-2646dab892aa None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:05.402749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacc644-fe2a-438d-9a7c-2646dab892aa None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:05.403015 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacc644-fe2a-438d-9a7c-2646dab892aa None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:05.403670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceacc644-fe2a-438d-9a7c-2646dab892aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:05.403964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceacc644-fe2a-438d-9a7c-2646dab892aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:05.405367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceacc644-fe2a-438d-9a7c-2646dab892aa None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:05.407494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:05.408002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d4235-ca3a-4e07-a329-eb6c7058c4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:05.408631 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d4235-ca3a-4e07-a329-eb6c7058c4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:05.408880 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d4235-ca3a-4e07-a329-eb6c7058c4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:05.409151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d4235-ca3a-4e07-a329-eb6c7058c4e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:05.409818 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d4235-ca3a-4e07-a329-eb6c7058c4e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:05.410129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d4235-ca3a-4e07-a329-eb6c7058c4e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:05.411503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d4235-ca3a-4e07-a329-eb6c7058c4e4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:05.414836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 223/444] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:05.415660 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacc644-fe2a-438d-9a7c-2646dab892aa None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:05.428150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 222/445] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:05.428150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d4235-ca3a-4e07-a329-eb6c7058c4e4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:05.508596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:05.519187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacc644-fe2a-438d-9a7c-2646dab892aa None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:05.529260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d4235-ca3a-4e07-a329-eb6c7058c4e4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:05.556489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a39f02-d346-4f80-9ae1-bcd76c1fcd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:05.557130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a39f02-d346-4f80-9ae1-bcd76c1fcd9e None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:05.557489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a39f02-d346-4f80-9ae1-bcd76c1fcd9e None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:05.557660 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a39f02-d346-4f80-9ae1-bcd76c1fcd9e None admin] PATH_INFO: `/v3/groups/ce9fe4f82f1f43e3be83457d040809b3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:05.558988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a39f02-d346-4f80-9ae1-bcd76c1fcd9e None admin] RBAC: Authorizing `identity:get_group(group_id=ce9fe4f82f1f43e3be83457d040809b3)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:05.560678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a39f02-d346-4f80-9ae1-bcd76c1fcd9e None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:05.562536 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 224/446] 10.222.0.62 () {56 vars in 1180 bytes} [Fri Sep 9 11:53:05 2022] GET /identity/v3/groups/ce9fe4f82f1f43e3be83457d040809b3 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:05.563695 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a39f02-d346-4f80-9ae1-bcd76c1fcd9e None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:05.567464 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadb7e2-3415-436c-88b7-9c0c202efe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:05.568489 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadb7e2-3415-436c-88b7-9c0c202efe1c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:05.568968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadb7e2-3415-436c-88b7-9c0c202efe1c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:05.569448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadb7e2-3415-436c-88b7-9c0c202efe1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:05.636525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:05.643166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadb7e2-3415-436c-88b7-9c0c202efe1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:05.644593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadb7e2-3415-436c-88b7-9c0c202efe1c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:05.672396 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a39f02-d346-4f80-9ae1-bcd76c1fcd9e None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:05.706594 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f4d56-f0b3-4c37-b301-96ae2be96846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:05.707117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f4d56-f0b3-4c37-b301-96ae2be96846 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:05.707396 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f4d56-f0b3-4c37-b301-96ae2be96846 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:05.707664 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f4d56-f0b3-4c37-b301-96ae2be96846 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:05.722536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:05.743385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 223/447] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:53:05.744468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadb7e2-3415-436c-88b7-9c0c202efe1c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:05.776813 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f4d56-f0b3-4c37-b301-96ae2be96846 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:05.778310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f4d56-f0b3-4c37-b301-96ae2be96846 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:05.843332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:05.843754 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadb7e2-3415-436c-88b7-9c0c202efe1c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:05.878482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e6a75-a461-4df0-a142-2ccb4761f128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:05.879084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e6a75-a461-4df0-a142-2ccb4761f128 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:05.879359 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e6a75-a461-4df0-a142-2ccb4761f128 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:05.879838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e6a75-a461-4df0-a142-2ccb4761f128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:05.879947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 225/448] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:53:05.880370 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e6a75-a461-4df0-a142-2ccb4761f128 None admin] RBAC: Adding query filter params (group.id=ce9fe4f82f1f43e3be83457d040809b3, role.id=e385f7d11fac4932a910ec0ef95e2a43, scope.project.id=3a96e4f9e7134925af6e684f79748f71) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:05.880587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f4d56-f0b3-4c37-b301-96ae2be96846 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:05.880680 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e6a75-a461-4df0-a142-2ccb4761f128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:05.882095 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e6a75-a461-4df0-a142-2ccb4761f128 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:05.888759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 224/449] 10.222.0.62 () {56 vars in 1376 bytes} [Fri Sep 9 11:53:05 2022] GET /identity/v3/role_assignments?group.id=ce9fe4f82f1f43e3be83457d040809b3&scope.project.id=3a96e4f9e7134925af6e684f79748f71&role.id=e385f7d11fac4932a910ec0ef95e2a43 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:05.889679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e6a75-a461-4df0-a142-2ccb4761f128 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:05.949085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:06.015850 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e6a75-a461-4df0-a142-2ccb4761f128 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:06.020789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f4d56-f0b3-4c37-b301-96ae2be96846 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:06.054942 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfbe11-96ff-4b2b-8f95-825c3c3d61d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:06.055587 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfbe11-96ff-4b2b-8f95-825c3c3d61d2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:06.055873 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfbe11-96ff-4b2b-8f95-825c3c3d61d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:06.056140 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfbe11-96ff-4b2b-8f95-825c3c3d61d2 None admin] PATH_INFO: `/v3/projects/fe7d1fa6c42548518e72832336a573e5` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:06.057326 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfbe11-96ff-4b2b-8f95-825c3c3d61d2 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7d1fa6c42548518e72832336a573e5)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:06.058828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfbe11-96ff-4b2b-8f95-825c3c3d61d2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:06.060600 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f05be-85d9-4a20-b98f-86be90fbee2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:06.061245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f05be-85d9-4a20-b98f-86be90fbee2b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:06.061589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f05be-85d9-4a20-b98f-86be90fbee2b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:06.061888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f05be-85d9-4a20-b98f-86be90fbee2b None admin] PATH_INFO: `/v3/projects/3a96e4f9e7134925af6e684f79748f71` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:06.063065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f05be-85d9-4a20-b98f-86be90fbee2b None admin] RBAC: Authorizing `identity:get_project(project_id=3a96e4f9e7134925af6e684f79748f71)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:06.064500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f05be-85d9-4a20-b98f-86be90fbee2b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:06.065208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 225/450] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:53:05 2022] GET /identity/v3/projects/fe7d1fa6c42548518e72832336a573e5 => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:06.066096 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfbe11-96ff-4b2b-8f95-825c3c3d61d2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:06.066309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 226/451] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:53:05 2022] GET /identity/v3/projects/3a96e4f9e7134925af6e684f79748f71 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:06.072573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f05be-85d9-4a20-b98f-86be90fbee2b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:06.116382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:06.188781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfbe11-96ff-4b2b-8f95-825c3c3d61d2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:06.192655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f05be-85d9-4a20-b98f-86be90fbee2b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:06.204531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:06.234725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1139f1-9011-40a4-aa58-6c15a4fe0a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:06.235365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1139f1-9011-40a4-aa58-6c15a4fe0a25 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:06.235644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1139f1-9011-40a4-aa58-6c15a4fe0a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:06.235920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1139f1-9011-40a4-aa58-6c15a4fe0a25 None admin] PATH_INFO: `/v3/groups/5deb34b4087e46c4a4cbe16c7fe15338` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:06.237040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1139f1-9011-40a4-aa58-6c15a4fe0a25 None admin] RBAC: Authorizing `identity:get_group(group_id=5deb34b4087e46c4a4cbe16c7fe15338)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:06.238513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1139f1-9011-40a4-aa58-6c15a4fe0a25 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:06.240094 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 226/452] 10.222.0.62 () {56 vars in 1180 bytes} [Fri Sep 9 11:53:06 2022] GET /identity/v3/groups/5deb34b4087e46c4a4cbe16c7fe15338 => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:06.243581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae970a-e98e-44da-91c3-f755b9066bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:06.246753 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1139f1-9011-40a4-aa58-6c15a4fe0a25 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:06.257700 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae970a-e98e-44da-91c3-f755b9066bee None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:06.258033 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae970a-e98e-44da-91c3-f755b9066bee None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:06.258302 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae970a-e98e-44da-91c3-f755b9066bee None admin] PATH_INFO: `/v3/groups/ce9fe4f82f1f43e3be83457d040809b3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:06.259557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae970a-e98e-44da-91c3-f755b9066bee None admin] RBAC: Authorizing `identity:get_group(group_id=ce9fe4f82f1f43e3be83457d040809b3)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:06.261007 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae970a-e98e-44da-91c3-f755b9066bee None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:06.263761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 227/453] 10.222.0.62 () {56 vars in 1180 bytes} [Fri Sep 9 11:53:06 2022] GET /identity/v3/groups/ce9fe4f82f1f43e3be83457d040809b3 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:06.270203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae970a-e98e-44da-91c3-f755b9066bee None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:06.362219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1139f1-9011-40a4-aa58-6c15a4fe0a25 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:06.384951 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae970a-e98e-44da-91c3-f755b9066bee None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:06.386471 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:06.397413 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cc770-6396-4c1a-a775-c1af73b80d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:06.398026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cc770-6396-4c1a-a775-c1af73b80d20 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:06.398827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cc770-6396-4c1a-a775-c1af73b80d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:06.399099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cc770-6396-4c1a-a775-c1af73b80d20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:06.399822 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350cc770-6396-4c1a-a775-c1af73b80d20 None admin] RBAC: Adding query filter params (group.id=5deb34b4087e46c4a4cbe16c7fe15338, role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=fe7d1fa6c42548518e72832336a573e5) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:06.400127 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350cc770-6396-4c1a-a775-c1af73b80d20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:06.401545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350cc770-6396-4c1a-a775-c1af73b80d20 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:06.407335 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 227/454] 10.222.0.62 () {56 vars in 1376 bytes} [Fri Sep 9 11:53:06 2022] GET /identity/v3/role_assignments?group.id=5deb34b4087e46c4a4cbe16c7fe15338&scope.project.id=fe7d1fa6c42548518e72832336a573e5&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:06.421754 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c15749-6a2a-4409-973d-e50e7e398843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:06.422419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c15749-6a2a-4409-973d-e50e7e398843 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:06.422693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c15749-6a2a-4409-973d-e50e7e398843 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:06.422960 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c15749-6a2a-4409-973d-e50e7e398843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:06.423705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c15749-6a2a-4409-973d-e50e7e398843 None admin] RBAC: Adding query filter params (group.id=ce9fe4f82f1f43e3be83457d040809b3, role.id=d43148a29472443ba7dfe2424ec60baf, scope.project.id=3a96e4f9e7134925af6e684f79748f71) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:06.424014 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c15749-6a2a-4409-973d-e50e7e398843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:06.425491 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c15749-6a2a-4409-973d-e50e7e398843 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:06.432227 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 228/455] 10.222.0.62 () {56 vars in 1376 bytes} [Fri Sep 9 11:53:06 2022] GET /identity/v3/role_assignments?group.id=ce9fe4f82f1f43e3be83457d040809b3&scope.project.id=3a96e4f9e7134925af6e684f79748f71&role.id=d43148a29472443ba7dfe2424ec60baf => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:09.210936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673d316-ff51-4f5c-9190-4a763d3b1b57 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:09.211459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673d316-ff51-4f5c-9190-4a763d3b1b57 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:09.211459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673d316-ff51-4f5c-9190-4a763d3b1b57 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:09.213196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 228/456] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:09.218817 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f00d86-ca93-4283-981a-84f8175a8f84 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:09.225387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f00d86-ca93-4283-981a-84f8175a8f84 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:09.225887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f00d86-ca93-4283-981a-84f8175a8f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:09.252087 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a9f00d86-ca93-4283-981a-84f8175a8f84 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:09.253637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a9f00d86-ca93-4283-981a-84f8175a8f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:09.342435 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 229/457] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:09.361425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf14bdc-5f7a-44c9-8671-5f9a531677d6 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:09.362028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf14bdc-5f7a-44c9-8671-5f9a531677d6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:09.362522 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf14bdc-5f7a-44c9-8671-5f9a531677d6 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:09.370399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 229/458] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:09.372684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459be3b-37e1-4075-9661-a1467d951616 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:09.372973 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459be3b-37e1-4075-9661-a1467d951616 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:09.373247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459be3b-37e1-4075-9661-a1467d951616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:09.401342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-7459be3b-37e1-4075-9661-a1467d951616 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:09.402599 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7459be3b-37e1-4075-9661-a1467d951616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:09.500556 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 230/459] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:09.511035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbda89-ec97-4b54-9d8c-a14ac7693818 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:09.511561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbda89-ec97-4b54-9d8c-a14ac7693818 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:09.512025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbda89-ec97-4b54-9d8c-a14ac7693818 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:09.514698 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 230/460] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:09.518825 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b619f-ba8e-4d05-935e-1ccce53831bf None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:09.519107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b619f-ba8e-4d05-935e-1ccce53831bf None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:09.519382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b619f-ba8e-4d05-935e-1ccce53831bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:09.543103 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-8a0b619f-ba8e-4d05-935e-1ccce53831bf None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:09.544308 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-8a0b619f-ba8e-4d05-935e-1ccce53831bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:09.570463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:09.632960 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 231/461] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:09.693688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbda89-ec97-4b54-9d8c-a14ac7693818 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:09.754388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:09.822849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbda89-ec97-4b54-9d8c-a14ac7693818 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:09.870838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3c5ce-b444-4955-843a-eecec905d7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:09.871487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3c5ce-b444-4955-843a-eecec905d7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:09.871765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3c5ce-b444-4955-843a-eecec905d7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:09.872036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3c5ce-b444-4955-843a-eecec905d7d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:09.883635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3c5ce-b444-4955-843a-eecec905d7d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:09.885284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3c5ce-b444-4955-843a-eecec905d7d9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:09.899259 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b619f-ba8e-4d05-935e-1ccce53831bf None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-e5a3c5ce-b444-4955-843a-eecec905d7d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:09.910625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:09.911551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:09.917646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:09.917646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:09.917646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:09.917646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 231/462] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:09.920646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3c5ce-b444-4955-843a-eecec905d7d9 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:10.029585 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:10.037185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3c5ce-b444-4955-843a-eecec905d7d9 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:10.059769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b619f-ba8e-4d05-935e-1ccce53831bf None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:10.076850 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae7c86-16a4-460b-a053-336d73cf4486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:10.077789 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae7c86-16a4-460b-a053-336d73cf4486 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:10.078267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae7c86-16a4-460b-a053-336d73cf4486 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:10.078715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae7c86-16a4-460b-a053-336d73cf4486 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:10.079689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae7c86-16a4-460b-a053-336d73cf4486 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:10.080594 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae7c86-16a4-460b-a053-336d73cf4486 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:10.082764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae7c86-16a4-460b-a053-336d73cf4486 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:10.100364 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 232/463] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:10.101620 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae7c86-16a4-460b-a053-336d73cf4486 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:10.106082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1aaa0b-92aa-43fa-81d3-6082b542e66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:10.107040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1aaa0b-92aa-43fa-81d3-6082b542e66e None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:10.107541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1aaa0b-92aa-43fa-81d3-6082b542e66e None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:10.108056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1aaa0b-92aa-43fa-81d3-6082b542e66e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:10.119786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1aaa0b-92aa-43fa-81d3-6082b542e66e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:10.121571 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1aaa0b-92aa-43fa-81d3-6082b542e66e None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-1c1aaa0b-92aa-43fa-81d3-6082b542e66e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:10.139601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:10.140702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:10.141613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:10.141613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:10.141613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:10.144177 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 232/464] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:10.145266 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1aaa0b-92aa-43fa-81d3-6082b542e66e None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:10.165642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:10.253945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae7c86-16a4-460b-a053-336d73cf4486 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:10.288208 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1aaa0b-92aa-43fa-81d3-6082b542e66e None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:10.297406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e086a-2ac4-4c78-9eea-f3cc9a9d08d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:10.298423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e086a-2ac4-4c78-9eea-f3cc9a9d08d9 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:10.298906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e086a-2ac4-4c78-9eea-f3cc9a9d08d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:10.299402 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e086a-2ac4-4c78-9eea-f3cc9a9d08d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:10.310872 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e086a-2ac4-4c78-9eea-f3cc9a9d08d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:10.312610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e086a-2ac4-4c78-9eea-f3cc9a9d08d9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-e10e086a-2ac4-4c78-9eea-f3cc9a9d08d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:10.333692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:10.334688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:10.335548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:10.335548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:10.335548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:10.336257 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61318cd-01f6-4215-a83a-d52e0930324f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:10.336796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61318cd-01f6-4215-a83a-d52e0930324f None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:10.336937 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61318cd-01f6-4215-a83a-d52e0930324f None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:10.337205 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61318cd-01f6-4215-a83a-d52e0930324f None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:10.337853 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61318cd-01f6-4215-a83a-d52e0930324f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:10.338291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 233/465] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:10.338955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e086a-2ac4-4c78-9eea-f3cc9a9d08d9 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:10.339482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61318cd-01f6-4215-a83a-d52e0930324f None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:10.352082 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:10.373726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 233/466] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:53:10 2022] POST /identity/v3/users => generated 246 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:10.373726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfd41a-265b-4fcd-89b0-e1af312f3317 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:10.373726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfd41a-265b-4fcd-89b0-e1af312f3317 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:10.373726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfd41a-265b-4fcd-89b0-e1af312f3317 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:10.373726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 234/467] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:10.373726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfd41a-265b-4fcd-89b0-e1af312f3317 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:10.471247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:10.473507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e086a-2ac4-4c78-9eea-f3cc9a9d08d9 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:10.515948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92017890-5696-4fee-beb7-d8085e5950de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:10.516362 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92017890-5696-4fee-beb7-d8085e5950de None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:10.516641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92017890-5696-4fee-beb7-d8085e5950de None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:10.516916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92017890-5696-4fee-beb7-d8085e5950de None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:10.517574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92017890-5696-4fee-beb7-d8085e5950de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:10.517971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92017890-5696-4fee-beb7-d8085e5950de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:10.519298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92017890-5696-4fee-beb7-d8085e5950de None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:10.525802 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfd41a-265b-4fcd-89b0-e1af312f3317 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:10.534265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 234/468] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:10.535612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e863ea-3c89-42d4-831f-14c21ab9ccf0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:10.535895 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e863ea-3c89-42d4-831f-14c21ab9ccf0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:10.536166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e863ea-3c89-42d4-831f-14c21ab9ccf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:10.559926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a6e863ea-3c89-42d4-831f-14c21ab9ccf0 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:10.560318 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a6e863ea-3c89-42d4-831f-14c21ab9ccf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:10.568309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73249f-f2ec-4d6a-a683-5c0163c05d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:10.568929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73249f-f2ec-4d6a-a683-5c0163c05d74 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:10.569208 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73249f-f2ec-4d6a-a683-5c0163c05d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:10.569482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73249f-f2ec-4d6a-a683-5c0163c05d74 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:10.571599 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73249f-f2ec-4d6a-a683-5c0163c05d74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:10.571918 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73249f-f2ec-4d6a-a683-5c0163c05d74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:10.573357 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73249f-f2ec-4d6a-a683-5c0163c05d74 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:10.592837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 235/469] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:10.593496 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73249f-f2ec-4d6a-a683-5c0163c05d74 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:10.641058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:10.651968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 235/470] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:10.653047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e863ea-3c89-42d4-831f-14c21ab9ccf0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:10.724057 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73249f-f2ec-4d6a-a683-5c0163c05d74 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:10.760867 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:10.784341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d47ddb-f15b-4d3b-b15b-55b4a13c0433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:10.784953 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d47ddb-f15b-4d3b-b15b-55b4a13c0433 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:10.785228 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d47ddb-f15b-4d3b-b15b-55b4a13c0433 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:10.785497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d47ddb-f15b-4d3b-b15b-55b4a13c0433 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:10.786170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d47ddb-f15b-4d3b-b15b-55b4a13c0433 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:10.787625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d47ddb-f15b-4d3b-b15b-55b4a13c0433 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:10.816396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e863ea-3c89-42d4-831f-14c21ab9ccf0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:10.837617 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 236/471] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:53:10 2022] POST /identity/v3/users => generated 248 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:10.859657 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7be8b-fa9f-43a0-a83b-668d34ac3c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:10.860667 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7be8b-fa9f-43a0-a83b-668d34ac3c3d None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:10.861174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7be8b-fa9f-43a0-a83b-668d34ac3c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:10.861748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7be8b-fa9f-43a0-a83b-668d34ac3c3d None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:10.862682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7be8b-fa9f-43a0-a83b-668d34ac3c3d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:10.864462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7be8b-fa9f-43a0-a83b-668d34ac3c3d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:10.893881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 236/472] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:53:10 2022] POST /identity/v3/users => generated 248 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:10.966251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d47ddb-f15b-4d3b-b15b-55b4a13c0433 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:11.122301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d47ddb-f15b-4d3b-b15b-55b4a13c0433 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:11.162791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fedba8-e891-46f0-8c96-23502e755be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:11.163437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fedba8-e891-46f0-8c96-23502e755be7 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:11.163797 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fedba8-e891-46f0-8c96-23502e755be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:11.164067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fedba8-e891-46f0-8c96-23502e755be7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:11.176755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fedba8-e891-46f0-8c96-23502e755be7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:11.178263 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fedba8-e891-46f0-8c96-23502e755be7 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:11.191329 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dde763-9d63-4678-acd9-0083c1797c2e None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:11.191641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dde763-9d63-4678-acd9-0083c1797c2e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:11.192019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dde763-9d63-4678-acd9-0083c1797c2e None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-d4fedba8-e891-46f0-8c96-23502e755be7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:11.194538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:11.195492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:11.196864 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:11.196864 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:11.196864 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:11.196864 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 237/473] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:11.197272 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 237/474] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:11.202080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fedba8-e891-46f0-8c96-23502e755be7 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:11.202971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ccd9d-5fe8-4c9f-9fc0-25b72640cf9e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:11.203244 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ccd9d-5fe8-4c9f-9fc0-25b72640cf9e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:11.203579 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ccd9d-5fe8-4c9f-9fc0-25b72640cf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:11.234944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-6c5ccd9d-5fe8-4c9f-9fc0-25b72640cf9e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:11.236147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6c5ccd9d-5fe8-4c9f-9fc0-25b72640cf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:11.249517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:11.300887 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:11.329206 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 238/475] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:11.341741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fedba8-e891-46f0-8c96-23502e755be7 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:11.378522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2bd92-380f-4b92-b224-8c4208b1a8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:11.379057 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2bd92-380f-4b92-b224-8c4208b1a8ae None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:11.379336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2bd92-380f-4b92-b224-8c4208b1a8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:11.379609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2bd92-380f-4b92-b224-8c4208b1a8ae None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:11.380325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2bd92-380f-4b92-b224-8c4208b1a8ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:11.380634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2bd92-380f-4b92-b224-8c4208b1a8ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:11.382032 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2bd92-380f-4b92-b224-8c4208b1a8ae None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:11.395991 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 238/476] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:11.402228 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ccd9d-5fe8-4c9f-9fc0-25b72640cf9e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:11.544494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ccd9d-5fe8-4c9f-9fc0-25b72640cf9e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:11.584767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abdf4d-28ab-48c7-9dcf-dd44bf9d0cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:11.585866 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abdf4d-28ab-48c7-9dcf-dd44bf9d0cea None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:11.586384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abdf4d-28ab-48c7-9dcf-dd44bf9d0cea None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:11.586884 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abdf4d-28ab-48c7-9dcf-dd44bf9d0cea None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:11.587736 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abdf4d-28ab-48c7-9dcf-dd44bf9d0cea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:11.589374 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abdf4d-28ab-48c7-9dcf-dd44bf9d0cea None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:11.612502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 239/477] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:53:11 2022] POST /identity/v3/users => generated 249 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:11.755061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2bd92-380f-4b92-b224-8c4208b1a8ae None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:11.891357 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:11.902452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2bd92-380f-4b92-b224-8c4208b1a8ae None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:11.952581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67122d62-403a-4376-a142-7f4bb7814226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:11.954337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67122d62-403a-4376-a142-7f4bb7814226 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:11.954337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67122d62-403a-4376-a142-7f4bb7814226 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:11.954337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67122d62-403a-4376-a142-7f4bb7814226 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:11.954568 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67122d62-403a-4376-a142-7f4bb7814226 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:11.955836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67122d62-403a-4376-a142-7f4bb7814226 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:11.964782 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.sql.core [None req-67122d62-403a-4376-a142-7f4bb7814226 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 11:53:11.964782 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 11:53:11.964782 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [parameters: {'id': '40ac632e845345338986270db6818aad', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 11:53:11.964782 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81447) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 11:53:11.981595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-67122d62-403a-4376-a142-7f4bb7814226 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 11:53:11.981595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:11.981595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 11:53:11.981595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 11:53:11.981595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 11:53:11.981595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 11:53:11.981595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 11:53:11.981595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 11:53:11.981595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 11:53:11.981595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application conn.query(q) Sep 09 11:53:11.981595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 11:53:11.981595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 11:53:11.981595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 11:53:11.981595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application result.read() Sep 09 11:53:11.981595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return ref.to_dict() Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 11:53:11.982558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application next(self.gen) Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application yield resource Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application next(self.gen) Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise self.value Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application session.commit() Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 11:53:11.984171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self.session.flush() Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application compat.raise_( Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 11:53:11.985373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 11:53:11.986757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application conn.query(q) Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application result.read() Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 11:53:11.991576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [parameters: {'id': '40ac632e845345338986270db6818aad', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:11.994223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 11:53:11.996289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:12.001210 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 239/478] 10.222.0.62 () {60 vars in 1131 bytes} [Fri Sep 9 11:53:11 2022] POST /identity/v3/roles => generated 144 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:12.001371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abdf4d-28ab-48c7-9dcf-dd44bf9d0cea None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:12.046848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:12.120418 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abdf4d-28ab-48c7-9dcf-dd44bf9d0cea None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:12.159491 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a09656-fb03-493a-abe3-d532bf33f446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:12.160140 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a09656-fb03-493a-abe3-d532bf33f446 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:12.160408 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a09656-fb03-493a-abe3-d532bf33f446 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:12.160683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a09656-fb03-493a-abe3-d532bf33f446 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:12.173628 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a09656-fb03-493a-abe3-d532bf33f446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-10a09656-fb03-493a-abe3-d532bf33f446 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:12.181163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:12.182421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:12.183249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:12.183249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:12.183249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:12.183249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:12.183249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:12.183249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:12.183249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:12.183249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:12.183249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:12.183249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:12.183249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:53:12.183249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:12.183249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 240/479] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:53:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:12.186965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67122d62-403a-4376-a142-7f4bb7814226 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:12.308401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67122d62-403a-4376-a142-7f4bb7814226 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:12.346004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe619efa-fcf2-4187-81aa-a75bda4edf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:12.346674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe619efa-fcf2-4187-81aa-a75bda4edf2f None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:12.346929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe619efa-fcf2-4187-81aa-a75bda4edf2f None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:12.347204 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe619efa-fcf2-4187-81aa-a75bda4edf2f None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:12.347893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe619efa-fcf2-4187-81aa-a75bda4edf2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:12.348197 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe619efa-fcf2-4187-81aa-a75bda4edf2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:12.349619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe619efa-fcf2-4187-81aa-a75bda4edf2f None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:12.360354 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 240/480] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:13.139173 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e681a-d76f-4016-97c5-7dd66f1c311e None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:13.139721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e681a-d76f-4016-97c5-7dd66f1c311e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:13.139721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e681a-d76f-4016-97c5-7dd66f1c311e None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:13.141557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 241/481] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:13.147738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0e770-6e50-4088-82b8-005f24512e20 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:13.148128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0e770-6e50-4088-82b8-005f24512e20 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:13.148912 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0e770-6e50-4088-82b8-005f24512e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:13.168099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-fcb0e770-6e50-4088-82b8-005f24512e20 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:13.169313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-fcb0e770-6e50-4088-82b8-005f24512e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:13.184728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:13.248115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 241/482] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:13.610588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e681a-d76f-4016-97c5-7dd66f1c311e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:13.743405 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:13.750758 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e681a-d76f-4016-97c5-7dd66f1c311e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:13.797188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a6b7c-dfd2-4613-bb8f-bfe391f7f4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:13.797828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a6b7c-dfd2-4613-bb8f-bfe391f7f4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:13.798102 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a6b7c-dfd2-4613-bb8f-bfe391f7f4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:13.798395 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a6b7c-dfd2-4613-bb8f-bfe391f7f4e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:13.814081 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a6b7c-dfd2-4613-bb8f-bfe391f7f4e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-046a6b7c-dfd2-4613-bb8f-bfe391f7f4e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:13.818911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:13.820332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:13.821155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:13.821155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:13.821155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:13.821155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:13.821155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:13.821155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:13.821155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:13.821155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:13.821155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:13.821155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:13.821155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:13.821155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:13.821155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 242/483] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:13.827686 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0e770-6e50-4088-82b8-005f24512e20 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:13.963515 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0e770-6e50-4088-82b8-005f24512e20 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:14.019429 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99334ed6-570a-4680-82f5-f51f58258186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:14.020064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99334ed6-570a-4680-82f5-f51f58258186 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:14.020344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99334ed6-570a-4680-82f5-f51f58258186 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:14.020615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99334ed6-570a-4680-82f5-f51f58258186 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:14.021299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99334ed6-570a-4680-82f5-f51f58258186 None admin] RBAC: Adding query filter params (name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:14.021631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99334ed6-570a-4680-82f5-f51f58258186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:14.023652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99334ed6-570a-4680-82f5-f51f58258186 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:14.038662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 242/484] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:14.045184 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a6b7c-dfd2-4613-bb8f-bfe391f7f4e5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:14.135481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1309c1-93e4-408a-8c75-9dbfe24cd93c None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:14.135842 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1309c1-93e4-408a-8c75-9dbfe24cd93c None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:14.136019 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1309c1-93e4-408a-8c75-9dbfe24cd93c None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:14.137491 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 243/485] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:14.143245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1517606-49c0-4d9f-9d78-edd645a4da29 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:14.143245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1517606-49c0-4d9f-9d78-edd645a4da29 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:14.143492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1517606-49c0-4d9f-9d78-edd645a4da29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:14.180331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-b1517606-49c0-4d9f-9d78-edd645a4da29 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:14.182067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b1517606-49c0-4d9f-9d78-edd645a4da29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:14.189125 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:14.198307 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a6b7c-dfd2-4613-bb8f-bfe391f7f4e5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:14.252973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d794613-3ddf-4ae3-80e6-f53b8cd7b226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:14.253608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d794613-3ddf-4ae3-80e6-f53b8cd7b226 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:14.253887 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d794613-3ddf-4ae3-80e6-f53b8cd7b226 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:14.254196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d794613-3ddf-4ae3-80e6-f53b8cd7b226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:14.267697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d794613-3ddf-4ae3-80e6-f53b8cd7b226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:14.269685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d794613-3ddf-4ae3-80e6-f53b8cd7b226 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:14.284154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 244/486] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:53:14.285428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b4ad4-65aa-4200-acfb-2fb0b26d910b None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:14.285711 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b4ad4-65aa-4200-acfb-2fb0b26d910b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:14.285983 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b4ad4-65aa-4200-acfb-2fb0b26d910b None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-4d794613-3ddf-4ae3-80e6-f53b8cd7b226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:14.287690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:14.288625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:14.290071 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:14.290071 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:14.290071 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:14.290071 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 243/487] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:14.294861 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 245/488] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:14.299097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d794613-3ddf-4ae3-80e6-f53b8cd7b226 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:14.303824 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfe98d-62ce-4883-b9eb-0b2e4194956f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:14.304105 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfe98d-62ce-4883-b9eb-0b2e4194956f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:14.304433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfe98d-62ce-4883-b9eb-0b2e4194956f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:14.331585 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:14.333711 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-1adfe98d-62ce-4883-b9eb-0b2e4194956f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:14.334965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-1adfe98d-62ce-4883-b9eb-0b2e4194956f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:14.434586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 246/489] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:14.504900 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d794613-3ddf-4ae3-80e6-f53b8cd7b226 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:14.555929 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937fe5b-b084-4f87-8ce3-12133b39df25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:14.556789 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937fe5b-b084-4f87-8ce3-12133b39df25 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:14.557211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937fe5b-b084-4f87-8ce3-12133b39df25 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:14.557663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937fe5b-b084-4f87-8ce3-12133b39df25 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:14.558541 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937fe5b-b084-4f87-8ce3-12133b39df25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:14.559002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937fe5b-b084-4f87-8ce3-12133b39df25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:14.560717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937fe5b-b084-4f87-8ce3-12133b39df25 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:14.571304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a8003-75bb-4ff1-abf9-cdc1bbee6530 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:14.571599 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a8003-75bb-4ff1-abf9-cdc1bbee6530 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:14.571877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a8003-75bb-4ff1-abf9-cdc1bbee6530 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:14.574226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 247/490] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:14.582156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9f5f5-150f-4672-b75a-5725acb69d4e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:14.582156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9f5f5-150f-4672-b75a-5725acb69d4e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:14.582156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9f5f5-150f-4672-b75a-5725acb69d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:14.588126 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 244/491] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:14.593164 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937fe5b-b084-4f87-8ce3-12133b39df25 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:14.608040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a0d9f5f5-150f-4672-b75a-5725acb69d4e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:14.609843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a0d9f5f5-150f-4672-b75a-5725acb69d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:14.705870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:14.712094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 248/492] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:53:14.712094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9f5f5-150f-4672-b75a-5725acb69d4e None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:14.740217 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:14.776849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937fe5b-b084-4f87-8ce3-12133b39df25 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:14.824604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f0268-23e9-4488-b01c-7b5f5724aa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:14.825524 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f0268-23e9-4488-b01c-7b5f5724aa5b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:14.825962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f0268-23e9-4488-b01c-7b5f5724aa5b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:14.826404 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f0268-23e9-4488-b01c-7b5f5724aa5b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:14.836928 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f0268-23e9-4488-b01c-7b5f5724aa5b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:14.838619 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f0268-23e9-4488-b01c-7b5f5724aa5b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:14.839582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9f5f5-150f-4672-b75a-5725acb69d4e None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-e31f0268-23e9-4488-b01c-7b5f5724aa5b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:14.854917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:14.856079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:14.857156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:14.857156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:14.857156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:14.857156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:14.857156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:14.857156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:14.857156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:14.857156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:14.857156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:53:14.857156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:14.858900 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 245/493] 10.222.0.62 () {56 vars in 1093 bytes} [Fri Sep 9 11:53:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:14.859839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f0268-23e9-4488-b01c-7b5f5724aa5b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:14.884815 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824d5d8-2c58-46fd-aa63-9fb5e2580b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:14.885532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824d5d8-2c58-46fd-aa63-9fb5e2580b4d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:14.885814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824d5d8-2c58-46fd-aa63-9fb5e2580b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:14.886083 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824d5d8-2c58-46fd-aa63-9fb5e2580b4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:14.898158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824d5d8-2c58-46fd-aa63-9fb5e2580b4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-d824d5d8-2c58-46fd-aa63-9fb5e2580b4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:14.903378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:14.904476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:14.905393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:14.905393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:14.905393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:14.905393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:14.905393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:14.905393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:14.905393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:14.905393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:14.905393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:14.905393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:14.905393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:14.905393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:14.905393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 249/494] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:14.905393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824d5d8-2c58-46fd-aa63-9fb5e2580b4d None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:14.976113 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:14.985854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f0268-23e9-4488-b01c-7b5f5724aa5b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:15.025697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:15.030574 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824d5d8-2c58-46fd-aa63-9fb5e2580b4d None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:15.031423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ba48f-f1ae-4223-b0c5-eb36fcc68ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:15.032229 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ba48f-f1ae-4223-b0c5-eb36fcc68ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:15.032506 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ba48f-f1ae-4223-b0c5-eb36fcc68ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:15.033602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ba48f-f1ae-4223-b0c5-eb36fcc68ad4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:15.048275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ba48f-f1ae-4223-b0c5-eb36fcc68ad4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-e30ba48f-f1ae-4223-b0c5-eb36fcc68ad4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:15.054593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:15.055499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:15.056294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:15.056294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:15.056294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:15.056294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:15.056294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:15.056294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:15.056294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:15.056294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:15.056294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:15.056294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:15.056294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:15.056294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:15.058943 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 246/495] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:15.060251 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ba48f-f1ae-4223-b0c5-eb36fcc68ad4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:15.069282 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab4413-ca66-4b35-8ac0-d61cf0614f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:15.070466 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab4413-ca66-4b35-8ac0-d61cf0614f48 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:15.070748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab4413-ca66-4b35-8ac0-d61cf0614f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:15.071021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab4413-ca66-4b35-8ac0-d61cf0614f48 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:15.071761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab4413-ca66-4b35-8ac0-d61cf0614f48 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:15.072068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab4413-ca66-4b35-8ac0-d61cf0614f48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:15.073525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab4413-ca66-4b35-8ac0-d61cf0614f48 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:15.113342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 250/496] 10.222.0.62 () {56 vars in 1133 bytes} [Fri Sep 9 11:53:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:15.114095 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab4413-ca66-4b35-8ac0-d61cf0614f48 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:15.166452 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ba48f-f1ae-4223-b0c5-eb36fcc68ad4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:15.203671 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1cb18-a33a-4d47-9a24-6885e10ff607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:15.204294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1cb18-a33a-4d47-9a24-6885e10ff607 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:15.204553 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1cb18-a33a-4d47-9a24-6885e10ff607 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:15.204826 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1cb18-a33a-4d47-9a24-6885e10ff607 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:15.205492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1cb18-a33a-4d47-9a24-6885e10ff607 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:15.205798 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1cb18-a33a-4d47-9a24-6885e10ff607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:15.207194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1cb18-a33a-4d47-9a24-6885e10ff607 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:15.212764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab4413-ca66-4b35-8ac0-d61cf0614f48 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:15.215739 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 247/497] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:15.216694 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1cb18-a33a-4d47-9a24-6885e10ff607 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:15.231428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:15.248418 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d133ff-f355-4a9e-b7d8-fdb4d9736892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:15.249044 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d133ff-f355-4a9e-b7d8-fdb4d9736892 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:15.249323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d133ff-f355-4a9e-b7d8-fdb4d9736892 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:15.249589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d133ff-f355-4a9e-b7d8-fdb4d9736892 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:15.260560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d133ff-f355-4a9e-b7d8-fdb4d9736892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-e8d133ff-f355-4a9e-b7d8-fdb4d9736892 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:15.265433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:15.266527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:15.267334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:15.267334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:15.267334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:15.267334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:15.267334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:15.267334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:15.267334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:15.267334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:15.267334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:15.267334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:15.267334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:15.267334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:15.267334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 251/498] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:15.268254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c40e8-777d-4b2b-ae74-a3bcb5cebd69 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:15.268566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c40e8-777d-4b2b-ae74-a3bcb5cebd69 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:15.268789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c40e8-777d-4b2b-ae74-a3bcb5cebd69 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:15.270030 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 252/499] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:15.270751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c40e8-777d-4b2b-ae74-a3bcb5cebd69 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:15.281970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:15.328391 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1cb18-a33a-4d47-9a24-6885e10ff607 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:15.363212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e90d1-20b6-4408-b9bd-a5a03d82c197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:15.363813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e90d1-20b6-4408-b9bd-a5a03d82c197 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:15.364090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e90d1-20b6-4408-b9bd-a5a03d82c197 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:15.364360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e90d1-20b6-4408-b9bd-a5a03d82c197 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:15.365030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e90d1-20b6-4408-b9bd-a5a03d82c197 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:15.365327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e90d1-20b6-4408-b9bd-a5a03d82c197 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:15.367266 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e90d1-20b6-4408-b9bd-a5a03d82c197 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:15.372951 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c40e8-777d-4b2b-ae74-a3bcb5cebd69 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:15.379824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 248/500] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:15.379824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e90d1-20b6-4408-b9bd-a5a03d82c197 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:15.407465 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb80f6-ae9b-4376-99fd-d3758fcceb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:15.408067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb80f6-ae9b-4376-99fd-d3758fcceb1f None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:15.408346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb80f6-ae9b-4376-99fd-d3758fcceb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:15.408620 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb80f6-ae9b-4376-99fd-d3758fcceb1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:15.454763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:15.478050 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb80f6-ae9b-4376-99fd-d3758fcceb1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:15.479538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb80f6-ae9b-4376-99fd-d3758fcceb1f None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:15.479874 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e90d1-20b6-4408-b9bd-a5a03d82c197 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:15.495838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:15.513594 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8fc46-33ab-4bca-9ba1-75e084f18b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:15.514245 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8fc46-33ab-4bca-9ba1-75e084f18b91 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:15.514519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8fc46-33ab-4bca-9ba1-75e084f18b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:15.514787 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8fc46-33ab-4bca-9ba1-75e084f18b91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:15.525455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8fc46-33ab-4bca-9ba1-75e084f18b91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:15.526902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8fc46-33ab-4bca-9ba1-75e084f18b91 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-76e8fc46-33ab-4bca-9ba1-75e084f18b91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:15.541463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:15.542568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:15.543561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:15.543561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:15.543561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:15.543561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 249/501] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:15.543561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8fc46-33ab-4bca-9ba1-75e084f18b91 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:15.578535 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 253/502] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:53:15.579875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892e29c-0797-4c81-b212-ed860e67dca6 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:15.580230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892e29c-0797-4c81-b212-ed860e67dca6 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:15.580579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892e29c-0797-4c81-b212-ed860e67dca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:15.598595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-4892e29c-0797-4c81-b212-ed860e67dca6 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:15.599796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-4892e29c-0797-4c81-b212-ed860e67dca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:15.646635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8fc46-33ab-4bca-9ba1-75e084f18b91 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:15.666107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 254/503] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:15.666979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892e29c-0797-4c81-b212-ed860e67dca6 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:15.676977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:15.687187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00926e5d-70cb-49ed-8ce6-af8125bc399f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:15.687793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00926e5d-70cb-49ed-8ce6-af8125bc399f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:15.688065 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00926e5d-70cb-49ed-8ce6-af8125bc399f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:15.688331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00926e5d-70cb-49ed-8ce6-af8125bc399f None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:15.688988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00926e5d-70cb-49ed-8ce6-af8125bc399f None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:15.689384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00926e5d-70cb-49ed-8ce6-af8125bc399f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:15.690787 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00926e5d-70cb-49ed-8ce6-af8125bc399f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:15.704154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 250/504] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:15.704154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00926e5d-70cb-49ed-8ce6-af8125bc399f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:15.734902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:15.784065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892e29c-0797-4c81-b212-ed860e67dca6 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:15.816609 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00926e5d-70cb-49ed-8ce6-af8125bc399f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:15.820882 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f6039-61e4-4d75-a079-d8dee0fd0afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:15.821593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f6039-61e4-4d75-a079-d8dee0fd0afa None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:15.821877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f6039-61e4-4d75-a079-d8dee0fd0afa None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:15.822173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f6039-61e4-4d75-a079-d8dee0fd0afa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:15.833271 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f6039-61e4-4d75-a079-d8dee0fd0afa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:15.835323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f6039-61e4-4d75-a079-d8dee0fd0afa None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-fa8f6039-61e4-4d75-a079-d8dee0fd0afa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:15.851572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:15.853045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:15.853883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:15.853883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:15.853883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:15.853883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 255/505] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:15.853883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f6039-61e4-4d75-a079-d8dee0fd0afa None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:15.860772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adec74-c502-4b89-89fd-c67fed4bbf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:15.861391 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adec74-c502-4b89-89fd-c67fed4bbf83 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:15.861671 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adec74-c502-4b89-89fd-c67fed4bbf83 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:15.861942 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adec74-c502-4b89-89fd-c67fed4bbf83 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:15.862624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adec74-c502-4b89-89fd-c67fed4bbf83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:15.862927 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adec74-c502-4b89-89fd-c67fed4bbf83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:15.864280 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adec74-c502-4b89-89fd-c67fed4bbf83 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:15.877402 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 251/506] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:15.883315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adec74-c502-4b89-89fd-c67fed4bbf83 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:15.925747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:15.968416 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f6039-61e4-4d75-a079-d8dee0fd0afa None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:15.977347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:15.987646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adec74-c502-4b89-89fd-c67fed4bbf83 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:16.002740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773f7dd-ce2c-4995-bae7-8f0d112b9144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:16.003549 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773f7dd-ce2c-4995-bae7-8f0d112b9144 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:16.003838 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773f7dd-ce2c-4995-bae7-8f0d112b9144 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:16.004109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773f7dd-ce2c-4995-bae7-8f0d112b9144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:16.015383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773f7dd-ce2c-4995-bae7-8f0d112b9144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:16.016826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773f7dd-ce2c-4995-bae7-8f0d112b9144 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:16.021944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934b5ac-170c-4549-bb72-11a1604a6dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:16.022606 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934b5ac-170c-4549-bb72-11a1604a6dfb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:16.022908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934b5ac-170c-4549-bb72-11a1604a6dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:16.023706 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934b5ac-170c-4549-bb72-11a1604a6dfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-b773f7dd-ce2c-4995-bae7-8f0d112b9144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:16.031532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:16.032625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:16.033463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:16.033463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:16.033463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:16.033463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 256/507] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:16.033463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773f7dd-ce2c-4995-bae7-8f0d112b9144 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:16.038354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934b5ac-170c-4549-bb72-11a1604a6dfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:16.039883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934b5ac-170c-4549-bb72-11a1604a6dfb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-1934b5ac-170c-4549-bb72-11a1604a6dfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:16.055219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:16.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:16.057074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:16.057074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:16.057074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:16.057074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 252/508] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:16.057287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934b5ac-170c-4549-bb72-11a1604a6dfb None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:16.134954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773f7dd-ce2c-4995-bae7-8f0d112b9144 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:16.147444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:16.158341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934b5ac-170c-4549-bb72-11a1604a6dfb None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:16.169828 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31233ebd-7451-49ff-8029-9411fa669335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:16.170466 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31233ebd-7451-49ff-8029-9411fa669335 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:16.170742 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31233ebd-7451-49ff-8029-9411fa669335 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:16.171064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31233ebd-7451-49ff-8029-9411fa669335 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:16.171864 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31233ebd-7451-49ff-8029-9411fa669335 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:16.172162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31233ebd-7451-49ff-8029-9411fa669335 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:16.177371 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31233ebd-7451-49ff-8029-9411fa669335 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:16.197312 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 257/509] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:16.197312 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31233ebd-7451-49ff-8029-9411fa669335 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:16.205497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599accd-f171-496b-8d8d-7f940207ed55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:16.206153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599accd-f171-496b-8d8d-7f940207ed55 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:16.206439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599accd-f171-496b-8d8d-7f940207ed55 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:16.206711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599accd-f171-496b-8d8d-7f940207ed55 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:16.213961 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599accd-f171-496b-8d8d-7f940207ed55 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:16.215430 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599accd-f171-496b-8d8d-7f940207ed55 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-9599accd-f171-496b-8d8d-7f940207ed55 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:16.227225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:16.228317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:16.229236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:16.229236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:16.229236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:16.229236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:16.229236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:16.229236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:16.229236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:16.229236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:16.229236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 11:53:16.229236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:16.229236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 253/510] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:16.229236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599accd-f171-496b-8d8d-7f940207ed55 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:16.231776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:16.296678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31233ebd-7451-49ff-8029-9411fa669335 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:16.330614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599accd-f171-496b-8d8d-7f940207ed55 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:16.330820 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b09797-c36d-4f86-af6b-4636640ed54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:16.331383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b09797-c36d-4f86-af6b-4636640ed54d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:16.331661 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b09797-c36d-4f86-af6b-4636640ed54d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:16.331932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b09797-c36d-4f86-af6b-4636640ed54d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:16.341955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b09797-c36d-4f86-af6b-4636640ed54d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:16.343446 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b09797-c36d-4f86-af6b-4636640ed54d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-19b09797-c36d-4f86-af6b-4636640ed54d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:16.358892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:16.359976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:16.360809 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:16.360809 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:16.360809 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:16.360809 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 258/511] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:16.360809 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b09797-c36d-4f86-af6b-4636640ed54d None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:16.366393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852088db-cf0e-4a83-8067-fceaa586d7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:16.367558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852088db-cf0e-4a83-8067-fceaa586d7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:16.367834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852088db-cf0e-4a83-8067-fceaa586d7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:16.368106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852088db-cf0e-4a83-8067-fceaa586d7a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:16.368752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852088db-cf0e-4a83-8067-fceaa586d7a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:16.369056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852088db-cf0e-4a83-8067-fceaa586d7a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:16.370730 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852088db-cf0e-4a83-8067-fceaa586d7a4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:16.383794 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 254/512] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:16.384609 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852088db-cf0e-4a83-8067-fceaa586d7a4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:16.389550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:16.446868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:16.471643 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b09797-c36d-4f86-af6b-4636640ed54d None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:16.486609 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852088db-cf0e-4a83-8067-fceaa586d7a4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:16.505398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ffa2c-993b-42b8-90c3-2ef20a39a997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:16.505997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ffa2c-993b-42b8-90c3-2ef20a39a997 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:16.506299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ffa2c-993b-42b8-90c3-2ef20a39a997 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:16.506574 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ffa2c-993b-42b8-90c3-2ef20a39a997 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:16.507215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ffa2c-993b-42b8-90c3-2ef20a39a997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:16.508098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ffa2c-993b-42b8-90c3-2ef20a39a997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:16.509485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ffa2c-993b-42b8-90c3-2ef20a39a997 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:16.520884 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cd936-7561-4fc0-a0db-56b1ba5c35c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:16.521046 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 259/513] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:16.521517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cd936-7561-4fc0-a0db-56b1ba5c35c4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:16.521774 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ffa2c-993b-42b8-90c3-2ef20a39a997 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:16.521882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cd936-7561-4fc0-a0db-56b1ba5c35c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:16.522043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cd936-7561-4fc0-a0db-56b1ba5c35c4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:16.533247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813cd936-7561-4fc0-a0db-56b1ba5c35c4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:16.534852 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813cd936-7561-4fc0-a0db-56b1ba5c35c4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-813cd936-7561-4fc0-a0db-56b1ba5c35c4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:16.547839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:16.548746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:16.549624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:16.549624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:16.549624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:16.549624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:16.549624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:16.549624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:16.549624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:16.549624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:16.549624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:53:16.549624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:16.549624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 255/514] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:16.550221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cd936-7561-4fc0-a0db-56b1ba5c35c4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:16.611800 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:16.627745 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ffa2c-993b-42b8-90c3-2ef20a39a997 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:16.654190 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cd936-7561-4fc0-a0db-56b1ba5c35c4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:16.662775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a040e-57bb-4635-aada-557ee50032bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:16.663378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a040e-57bb-4635-aada-557ee50032bf None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:16.663711 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a040e-57bb-4635-aada-557ee50032bf None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:16.663977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a040e-57bb-4635-aada-557ee50032bf None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:16.664704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a040e-57bb-4635-aada-557ee50032bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:16.665005 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a040e-57bb-4635-aada-557ee50032bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:16.666467 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a040e-57bb-4635-aada-557ee50032bf None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:16.676279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:16.684324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 260/515] 10.222.0.62 () {56 vars in 1137 bytes} [Fri Sep 9 11:53:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:16.685065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a040e-57bb-4635-aada-557ee50032bf None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:16.692461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9985256-fa65-412f-9e82-319b7216416e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:16.693073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9985256-fa65-412f-9e82-319b7216416e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:16.693378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9985256-fa65-412f-9e82-319b7216416e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:16.693610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9985256-fa65-412f-9e82-319b7216416e None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:16.694280 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9985256-fa65-412f-9e82-319b7216416e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:16.694582 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9985256-fa65-412f-9e82-319b7216416e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:16.696475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9985256-fa65-412f-9e82-319b7216416e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:16.708363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 256/516] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:16.709541 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9985256-fa65-412f-9e82-319b7216416e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:16.783964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a040e-57bb-4635-aada-557ee50032bf None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:16.810572 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9985256-fa65-412f-9e82-319b7216416e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:16.817942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791f7ad-d974-4539-b0f4-d82d7ea47cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:16.818603 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791f7ad-d974-4539-b0f4-d82d7ea47cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:16.818880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791f7ad-d974-4539-b0f4-d82d7ea47cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:16.819148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791f7ad-d974-4539-b0f4-d82d7ea47cd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:16.826416 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791f7ad-d974-4539-b0f4-d82d7ea47cd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:16.827855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791f7ad-d974-4539-b0f4-d82d7ea47cd8 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:16.840073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-9791f7ad-d974-4539-b0f4-d82d7ea47cd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:16.840073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:16.840073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:16.840073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:16.840073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:16.840073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:16.840073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:16.840073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:16.840073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:16.840073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:16.840073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:16.840073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:16.840073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:16.840073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:16.840073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:16.841055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:16.841911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:16.841911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:16.841911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:16.841911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:16.841911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:16.841911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:16.841911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:16.841911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:16.845777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 261/517] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:16.846578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218e1de-ab75-47cb-ac07-46e979453e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:16.846700 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791f7ad-d974-4539-b0f4-d82d7ea47cd8 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:16.847212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218e1de-ab75-47cb-ac07-46e979453e48 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:16.847488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218e1de-ab75-47cb-ac07-46e979453e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:16.847757 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218e1de-ab75-47cb-ac07-46e979453e48 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:16.849226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:16.854954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8218e1de-ab75-47cb-ac07-46e979453e48 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:16.856396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8218e1de-ab75-47cb-ac07-46e979453e48 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-8218e1de-ab75-47cb-ac07-46e979453e48 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:16.868776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:16.869688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:16.871077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:16.871077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:16.871077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:16.871077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:16.871077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:16.871077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:16.871077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:16.871077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:16.871077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 11:53:16.871077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:16.871077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 257/518] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:16.871077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218e1de-ab75-47cb-ac07-46e979453e48 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:16.914253 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:16.952317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791f7ad-d974-4539-b0f4-d82d7ea47cd8 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:16.972608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218e1de-ab75-47cb-ac07-46e979453e48 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:16.985693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f3714-55a9-435b-a60d-f49af25df606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:16.986320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f3714-55a9-435b-a60d-f49af25df606 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:16.986600 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f3714-55a9-435b-a60d-f49af25df606 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:16.986869 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f3714-55a9-435b-a60d-f49af25df606 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:16.987598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f3714-55a9-435b-a60d-f49af25df606 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:16.987901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f3714-55a9-435b-a60d-f49af25df606 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:16.989335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f3714-55a9-435b-a60d-f49af25df606 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:17.006659 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bbf42-4f49-4a41-8ee0-a2f31b0983cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:17.007268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bbf42-4f49-4a41-8ee0-a2f31b0983cb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:17.008115 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bbf42-4f49-4a41-8ee0-a2f31b0983cb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:17.008386 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bbf42-4f49-4a41-8ee0-a2f31b0983cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:17.008516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 262/519] 10.222.0.62 () {56 vars in 1137 bytes} [Fri Sep 9 11:53:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:17.009396 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f3714-55a9-435b-a60d-f49af25df606 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:17.068898 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:17.081848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bbf42-4f49-4a41-8ee0-a2f31b0983cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:17.083335 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bbf42-4f49-4a41-8ee0-a2f31b0983cb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:17.111694 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f3714-55a9-435b-a60d-f49af25df606 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:17.125497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:17.146318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66101e0-0db5-43c8-a875-56e332e4dd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:17.146919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66101e0-0db5-43c8-a875-56e332e4dd0a None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:17.147203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66101e0-0db5-43c8-a875-56e332e4dd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:17.147471 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66101e0-0db5-43c8-a875-56e332e4dd0a None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:17.148176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66101e0-0db5-43c8-a875-56e332e4dd0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:17.149618 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66101e0-0db5-43c8-a875-56e332e4dd0a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:17.170895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 263/520] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:53:17 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:17.171653 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66101e0-0db5-43c8-a875-56e332e4dd0a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:17.189193 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 258/521] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:53:17.189193 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bbf42-4f49-4a41-8ee0-a2f31b0983cb None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:17.301300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bbf42-4f49-4a41-8ee0-a2f31b0983cb None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:17.307016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66101e0-0db5-43c8-a875-56e332e4dd0a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:17.325438 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:17.344848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7abbce-11b5-44ef-8e21-a37be9019ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:17.345640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7abbce-11b5-44ef-8e21-a37be9019ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:17.345753 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7abbce-11b5-44ef-8e21-a37be9019ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:17.346732 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7abbce-11b5-44ef-8e21-a37be9019ed6 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:17.346732 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7abbce-11b5-44ef-8e21-a37be9019ed6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:17.347043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7abbce-11b5-44ef-8e21-a37be9019ed6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:17.349158 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7abbce-11b5-44ef-8e21-a37be9019ed6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:17.349857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34b861-28b9-4583-ab05-162e2b4ac6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:17.350500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34b861-28b9-4583-ab05-162e2b4ac6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:17.351978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34b861-28b9-4583-ab05-162e2b4ac6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:17.351978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34b861-28b9-4583-ab05-162e2b4ac6e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:17.352261 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34b861-28b9-4583-ab05-162e2b4ac6e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:17.352561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34b861-28b9-4583-ab05-162e2b4ac6e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:17.353984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34b861-28b9-4583-ab05-162e2b4ac6e9 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:17.373754 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 264/522] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:17.374656 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34b861-28b9-4583-ab05-162e2b4ac6e9 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:17.384202 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:17.389179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 259/523] 10.222.0.62 () {56 vars in 1139 bytes} [Fri Sep 9 11:53:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:17.392751 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7abbce-11b5-44ef-8e21-a37be9019ed6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:17.482006 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34b861-28b9-4583-ab05-162e2b4ac6e9 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:17.505425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7abbce-11b5-44ef-8e21-a37be9019ed6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:17.517732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e2eef-ddee-40fa-92a8-b7028163860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:17.518371 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e2eef-ddee-40fa-92a8-b7028163860a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:17.518649 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e2eef-ddee-40fa-92a8-b7028163860a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:17.518917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e2eef-ddee-40fa-92a8-b7028163860a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:17.546081 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc5706-ed5f-43e9-907c-e16e8a2cf2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:17.547137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc5706-ed5f-43e9-907c-e16e8a2cf2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:17.547586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc5706-ed5f-43e9-907c-e16e8a2cf2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:17.548041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc5706-ed5f-43e9-907c-e16e8a2cf2c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:17.560128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc5706-ed5f-43e9-907c-e16e8a2cf2c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:17.562050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc5706-ed5f-43e9-907c-e16e8a2cf2c5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:17.564972 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-68dc5706-ed5f-43e9-907c-e16e8a2cf2c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:17.580020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:17.581147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:17.582263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:17.582263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:17.582263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:17.583671 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 260/524] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:17.584638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc5706-ed5f-43e9-907c-e16e8a2cf2c5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:17.591898 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e2eef-ddee-40fa-92a8-b7028163860a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:17.593369 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e2eef-ddee-40fa-92a8-b7028163860a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:17.643190 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:17.695789 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc5706-ed5f-43e9-907c-e16e8a2cf2c5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:17.696528 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 265/525] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:53:17.701276 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e2eef-ddee-40fa-92a8-b7028163860a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:17.734337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef5dc6-c6cd-41c3-af8b-db22ad086e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:17.735262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef5dc6-c6cd-41c3-af8b-db22ad086e9d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:17.735732 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef5dc6-c6cd-41c3-af8b-db22ad086e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:17.736190 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef5dc6-c6cd-41c3-af8b-db22ad086e9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:17.737099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef5dc6-c6cd-41c3-af8b-db22ad086e9d None admin] RBAC: Adding query filter params (role.id=035f7d9f445d406eb2e83ca90fc4d5ca, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=1b30e80c421245b5a1ed5849b28ea90e) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:17.737404 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef5dc6-c6cd-41c3-af8b-db22ad086e9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:17.738848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef5dc6-c6cd-41c3-af8b-db22ad086e9d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:17.754146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 261/526] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:17 2022] GET /identity/v3/role_assignments?user.id=1b30e80c421245b5a1ed5849b28ea90e&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=035f7d9f445d406eb2e83ca90fc4d5ca => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:17.754146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef5dc6-c6cd-41c3-af8b-db22ad086e9d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:17.801856 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:17.838378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e2eef-ddee-40fa-92a8-b7028163860a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:17.885171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb777b-a932-42d7-8f6e-3c9bb18e58b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:17.886470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb777b-a932-42d7-8f6e-3c9bb18e58b0 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:17.887048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb777b-a932-42d7-8f6e-3c9bb18e58b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:17.887469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb777b-a932-42d7-8f6e-3c9bb18e58b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:17.899313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef5dc6-c6cd-41c3-af8b-db22ad086e9d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:17.922209 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:17.948095 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcdd77-8dd3-413c-986b-673e6eb5ea42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:17.949062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcdd77-8dd3-413c-986b-673e6eb5ea42 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:17.949513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcdd77-8dd3-413c-986b-673e6eb5ea42 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:17.949953 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcdd77-8dd3-413c-986b-673e6eb5ea42 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:17.950816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dcdd77-8dd3-413c-986b-673e6eb5ea42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:17.951352 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dcdd77-8dd3-413c-986b-673e6eb5ea42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:17.953729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dcdd77-8dd3-413c-986b-673e6eb5ea42 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:17.971664 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 262/527] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:17.972371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcdd77-8dd3-413c-986b-673e6eb5ea42 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:18.004888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb777b-a932-42d7-8f6e-3c9bb18e58b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:18.006291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb777b-a932-42d7-8f6e-3c9bb18e58b0 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:18.090530 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:18.099002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcdd77-8dd3-413c-986b-673e6eb5ea42 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:18.129232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 266/528] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:53:18.130883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb777b-a932-42d7-8f6e-3c9bb18e58b0 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:18.144663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a7dfe-a0c6-4806-9702-f3784506961e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:18.144999 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7dfe-a0c6-4806-9702-f3784506961e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:18.145273 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7dfe-a0c6-4806-9702-f3784506961e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:18.145544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7dfe-a0c6-4806-9702-f3784506961e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:18.157445 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a7dfe-a0c6-4806-9702-f3784506961e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:18.159189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a7dfe-a0c6-4806-9702-f3784506961e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-877a7dfe-a0c6-4806-9702-f3784506961e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:18.176208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:18.177155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:18.178001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:18.178001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:18.178001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:18.178001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 263/529] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:18.183161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a7dfe-a0c6-4806-9702-f3784506961e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:18.219457 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:18.258982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb777b-a932-42d7-8f6e-3c9bb18e58b0 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:18.294271 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298c6b1-ce5a-432d-827d-34ce3c183ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:18.294901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298c6b1-ce5a-432d-827d-34ce3c183ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:18.295175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298c6b1-ce5a-432d-827d-34ce3c183ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:18.295439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298c6b1-ce5a-432d-827d-34ce3c183ee3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:18.303219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298c6b1-ce5a-432d-827d-34ce3c183ee3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:18.304934 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298c6b1-ce5a-432d-827d-34ce3c183ee3 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:18.305539 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a7dfe-a0c6-4806-9702-f3784506961e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:18.317393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-d298c6b1-ce5a-432d-827d-34ce3c183ee3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:18.317393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:18.317393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:18.317393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:18.317393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:18.317393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:18.317393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:18.317393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:18.317393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:18.317393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:18.317393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:18.317393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:18.317393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:18.317393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:18.317393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:18.318288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:18.319128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:18.319128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:18.319128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:18.319128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:18.319128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:18.319128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:18.319128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:18.319128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:18.324098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 267/530] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:18.324959 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298c6b1-ce5a-432d-827d-34ce3c183ee3 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:18.348877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1a80a-175f-4d5f-936d-8d00473739b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:18.349930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1a80a-175f-4d5f-936d-8d00473739b4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:18.350440 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1a80a-175f-4d5f-936d-8d00473739b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:18.350931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1a80a-175f-4d5f-936d-8d00473739b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:18.363999 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1a80a-175f-4d5f-936d-8d00473739b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:18.365837 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1a80a-175f-4d5f-936d-8d00473739b4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:18.366317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-e8a1a80a-175f-4d5f-936d-8d00473739b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:18.385687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:18.387119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:18.388141 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:18.388141 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:18.388141 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:18.391180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 264/531] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:18.392234 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1a80a-175f-4d5f-936d-8d00473739b4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:18.442483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298c6b1-ce5a-432d-827d-34ce3c183ee3 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:18.483219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a7823-eccf-4ab2-b19b-0082d78f1274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:18.484079 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a7823-eccf-4ab2-b19b-0082d78f1274 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:18.484353 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a7823-eccf-4ab2-b19b-0082d78f1274 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:18.484619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a7823-eccf-4ab2-b19b-0082d78f1274 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:18.485264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a7823-eccf-4ab2-b19b-0082d78f1274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:18.485560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a7823-eccf-4ab2-b19b-0082d78f1274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:18.487134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a7823-eccf-4ab2-b19b-0082d78f1274 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:18.495940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:18.514643 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 268/532] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:18.514643 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a7823-eccf-4ab2-b19b-0082d78f1274 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:18.524992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1a80a-175f-4d5f-936d-8d00473739b4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:18.564935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ea2a4-526d-4826-a773-bcb6781706f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:18.565984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ea2a4-526d-4826-a773-bcb6781706f3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:18.566569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ea2a4-526d-4826-a773-bcb6781706f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:18.567150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ea2a4-526d-4826-a773-bcb6781706f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:18.568036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ea2a4-526d-4826-a773-bcb6781706f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:18.568531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ea2a4-526d-4826-a773-bcb6781706f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:18.570301 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ea2a4-526d-4826-a773-bcb6781706f3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:18.593539 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 265/533] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:18.594718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ea2a4-526d-4826-a773-bcb6781706f3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:18.619664 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:18.626289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a7823-eccf-4ab2-b19b-0082d78f1274 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:18.664436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf09db3-e7c9-4c42-8ac5-6ba8e743e284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:18.665144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf09db3-e7c9-4c42-8ac5-6ba8e743e284 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:18.665429 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf09db3-e7c9-4c42-8ac5-6ba8e743e284 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:18.665695 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf09db3-e7c9-4c42-8ac5-6ba8e743e284 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:18.666572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf09db3-e7c9-4c42-8ac5-6ba8e743e284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:18.666765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf09db3-e7c9-4c42-8ac5-6ba8e743e284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:18.668127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf09db3-e7c9-4c42-8ac5-6ba8e743e284 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:18.685233 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 269/534] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:18.686238 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf09db3-e7c9-4c42-8ac5-6ba8e743e284 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:18.716612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ea2a4-526d-4826-a773-bcb6781706f3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:18.761424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6fd81-efaf-479f-b3b0-c50022f97e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:18.762477 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6fd81-efaf-479f-b3b0-c50022f97e79 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:18.762966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6fd81-efaf-479f-b3b0-c50022f97e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:18.763548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6fd81-efaf-479f-b3b0-c50022f97e79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:18.772459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6fd81-efaf-479f-b3b0-c50022f97e79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:18.775492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6fd81-efaf-479f-b3b0-c50022f97e79 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:18.779202 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:18.795458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-48d6fd81-efaf-479f-b3b0-c50022f97e79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:18.795458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:18.795458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:18.795458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:18.795458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:18.795458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:18.795458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:18.795458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:18.795458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:18.795458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:18.795458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:18.795458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:18.795458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:18.795458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:18.795458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:18.796380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:18.797219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:18.797219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:18.797219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:18.797219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:18.797219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:18.797219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:18.797219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:18.797219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:18.799641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 266/535] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:18.800791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6fd81-efaf-479f-b3b0-c50022f97e79 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:18.827343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf09db3-e7c9-4c42-8ac5-6ba8e743e284 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:18.872983 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad3b99-6672-4019-861e-855c4a328014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:18.873840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad3b99-6672-4019-861e-855c4a328014 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:18.874295 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad3b99-6672-4019-861e-855c4a328014 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:18.874775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad3b99-6672-4019-861e-855c4a328014 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:18.875679 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad3b99-6672-4019-861e-855c4a328014 None admin] RBAC: Adding query filter params (role.id=035f7d9f445d406eb2e83ca90fc4d5ca, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=2d477a49a1724b7e8a4699a5ebd04bde) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:18.876122 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad3b99-6672-4019-861e-855c4a328014 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:18.877997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad3b99-6672-4019-861e-855c4a328014 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:18.885552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 270/536] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:18 2022] GET /identity/v3/role_assignments?user.id=2d477a49a1724b7e8a4699a5ebd04bde&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=035f7d9f445d406eb2e83ca90fc4d5ca => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:18.887066 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad3b99-6672-4019-861e-855c4a328014 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:18.936031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:18.947145 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6fd81-efaf-479f-b3b0-c50022f97e79 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:19.047063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4b6b0-d959-4a49-9155-067937e01d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:19.047654 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4b6b0-d959-4a49-9155-067937e01d89 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:19.047934 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4b6b0-d959-4a49-9155-067937e01d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:19.048215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4b6b0-d959-4a49-9155-067937e01d89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:19.063337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4b6b0-d959-4a49-9155-067937e01d89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:19.064825 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4b6b0-d959-4a49-9155-067937e01d89 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:19.083388 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad3b99-6672-4019-861e-855c4a328014 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:19.092521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-6ae4b6b0-d959-4a49-9155-067937e01d89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:19.092521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:19.092521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:19.092521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:19.092521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:19.092521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:19.092521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:19.092521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:19.092521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:19.092521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:19.092521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:19.092521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:19.092521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:19.092521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:19.092521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:19.094194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:19.095117 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:19.095117 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:19.095117 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:19.095117 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:19.095117 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:19.095117 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:19.095117 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:19.095117 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:19.095117 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 267/537] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:19.103174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4b6b0-d959-4a49-9155-067937e01d89 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:19.154842 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2c1ba-eaba-41bd-9615-90052eb445cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:19.155454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2c1ba-eaba-41bd-9615-90052eb445cc None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:19.155732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2c1ba-eaba-41bd-9615-90052eb445cc None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:19.157284 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2c1ba-eaba-41bd-9615-90052eb445cc None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:19.159294 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2c1ba-eaba-41bd-9615-90052eb445cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:19.159609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2c1ba-eaba-41bd-9615-90052eb445cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:19.163231 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2c1ba-eaba-41bd-9615-90052eb445cc None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:19.187608 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 271/538] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:19.193031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2c1ba-eaba-41bd-9615-90052eb445cc None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:19.201301 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:19.248839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4b6b0-d959-4a49-9155-067937e01d89 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:19.297995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ae6cd-c49a-46fc-bf94-cf16423f800e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:19.298762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ae6cd-c49a-46fc-bf94-cf16423f800e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:19.299038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ae6cd-c49a-46fc-bf94-cf16423f800e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:19.299309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ae6cd-c49a-46fc-bf94-cf16423f800e None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:19.300000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4ae6cd-c49a-46fc-bf94-cf16423f800e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:19.300300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4ae6cd-c49a-46fc-bf94-cf16423f800e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:19.304950 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4ae6cd-c49a-46fc-bf94-cf16423f800e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:19.331707 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 268/539] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:19.340896 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ae6cd-c49a-46fc-bf94-cf16423f800e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:19.354462 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:19.392454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2c1ba-eaba-41bd-9615-90052eb445cc None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:19.467050 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f23c7-ef30-44e2-aec3-78beef634712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:19.467662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f23c7-ef30-44e2-aec3-78beef634712 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:19.467947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f23c7-ef30-44e2-aec3-78beef634712 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:19.468216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f23c7-ef30-44e2-aec3-78beef634712 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:19.468957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f23c7-ef30-44e2-aec3-78beef634712 None admin] RBAC: Adding query filter params (role.id=035f7d9f445d406eb2e83ca90fc4d5ca, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=a25411d51a524f4b9f0ff846a0b29b6f) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:19.469265 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f23c7-ef30-44e2-aec3-78beef634712 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:19.474415 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f23c7-ef30-44e2-aec3-78beef634712 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:19.485366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 272/540] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:19 2022] GET /identity/v3/role_assignments?user.id=a25411d51a524f4b9f0ff846a0b29b6f&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=035f7d9f445d406eb2e83ca90fc4d5ca => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:19.518904 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ae6cd-c49a-46fc-bf94-cf16423f800e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:19.519731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d35bb-70d0-4519-af1b-c35504178e9d None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:19.520013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d35bb-70d0-4519-af1b-c35504178e9d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:19.520282 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d35bb-70d0-4519-af1b-c35504178e9d None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:19.521871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 273/541] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:53:19.534211 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14eab48-cab4-4c34-a16a-c7516c08ee38 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:19.534211 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14eab48-cab4-4c34-a16a-c7516c08ee38 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:19.534211 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14eab48-cab4-4c34-a16a-c7516c08ee38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:19.548192 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f14eab48-cab4-4c34-a16a-c7516c08ee38 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:19.549297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f14eab48-cab4-4c34-a16a-c7516c08ee38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:19.559175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:19.595572 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a6985-6589-4bf8-8fed-ae7a8cdd3709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:19.596199 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a6985-6589-4bf8-8fed-ae7a8cdd3709 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:19.596474 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a6985-6589-4bf8-8fed-ae7a8cdd3709 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:19.596742 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a6985-6589-4bf8-8fed-ae7a8cdd3709 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:19.597480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a6985-6589-4bf8-8fed-ae7a8cdd3709 None admin] RBAC: Adding query filter params (role.id=035f7d9f445d406eb2e83ca90fc4d5ca, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=d00634cdd3fb41e794d3af0b88ba7338) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:19.597783 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a6985-6589-4bf8-8fed-ae7a8cdd3709 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:19.599276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a6985-6589-4bf8-8fed-ae7a8cdd3709 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:19.613049 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 269/542] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:19 2022] GET /identity/v3/role_assignments?user.id=d00634cdd3fb41e794d3af0b88ba7338&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=035f7d9f445d406eb2e83ca90fc4d5ca => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:19.693158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 274/543] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:53:19.988853 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a6985-6589-4bf8-8fed-ae7a8cdd3709 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:20.131014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a6985-6589-4bf8-8fed-ae7a8cdd3709 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:20.192546 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404cca2-1e48-45fb-aa94-3f53f8c974fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:20.193200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404cca2-1e48-45fb-aa94-3f53f8c974fe None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:20.193476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404cca2-1e48-45fb-aa94-3f53f8c974fe None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:20.193740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404cca2-1e48-45fb-aa94-3f53f8c974fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:20.215640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404cca2-1e48-45fb-aa94-3f53f8c974fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-c404cca2-1e48-45fb-aa94-3f53f8c974fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:20.221215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:20.222107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:20.222940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:20.222940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:20.222940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:20.222940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:20.222940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:20.222940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:20.222940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:20.222940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:20.222940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:20.222940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:20.222940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:20.222940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:20.222940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 270/544] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:20.228270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969f74e-a85e-4c88-a643-715666ab5752 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:20.228723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969f74e-a85e-4c88-a643-715666ab5752 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:20.228855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969f74e-a85e-4c88-a643-715666ab5752 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:20.230441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 275/545] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:20.231368 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404cca2-1e48-45fb-aa94-3f53f8c974fe None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:20.250706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbcb09-4623-41b8-b4c6-7b480ebc7a61 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:20.251252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbcb09-4623-41b8-b4c6-7b480ebc7a61 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:20.251708 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbcb09-4623-41b8-b4c6-7b480ebc7a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:20.283625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-efbbcb09-4623-41b8-b4c6-7b480ebc7a61 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:20.284930 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-efbbcb09-4623-41b8-b4c6-7b480ebc7a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:20.293434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:20.357599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404cca2-1e48-45fb-aa94-3f53f8c974fe None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:20.409434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97a9d2-808f-44bc-8a4e-a9980a6dc9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:20.410203 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97a9d2-808f-44bc-8a4e-a9980a6dc9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:20.410556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97a9d2-808f-44bc-8a4e-a9980a6dc9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:20.410912 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97a9d2-808f-44bc-8a4e-a9980a6dc9d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:20.413981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97a9d2-808f-44bc-8a4e-a9980a6dc9d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:20.413981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97a9d2-808f-44bc-8a4e-a9980a6dc9d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:20.414202 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97a9d2-808f-44bc-8a4e-a9980a6dc9d4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:20.423720 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 271/546] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:20.429323 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97a9d2-808f-44bc-8a4e-a9980a6dc9d4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:20.447699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 276/547] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:20.541451 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97a9d2-808f-44bc-8a4e-a9980a6dc9d4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:20.561646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:20.583637 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362172b5-7583-42aa-92c4-7a35f9fb14de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:20.584280 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362172b5-7583-42aa-92c4-7a35f9fb14de None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:20.584560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362172b5-7583-42aa-92c4-7a35f9fb14de None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:20.584834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362172b5-7583-42aa-92c4-7a35f9fb14de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:20.600886 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362172b5-7583-42aa-92c4-7a35f9fb14de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:20.602455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362172b5-7583-42aa-92c4-7a35f9fb14de None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-362172b5-7583-42aa-92c4-7a35f9fb14de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:20.623765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:20.624716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:20.625586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:20.625586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:20.625586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:20.631323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbcb09-4623-41b8-b4c6-7b480ebc7a61 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:20.632667 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 272/548] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:20.643535 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:20.791569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbcb09-4623-41b8-b4c6-7b480ebc7a61 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:20.842696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362172b5-7583-42aa-92c4-7a35f9fb14de None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:20.843420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7c784-6860-4a45-81f1-195ce85a5b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:20.844909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7c784-6860-4a45-81f1-195ce85a5b94 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:20.845188 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7c784-6860-4a45-81f1-195ce85a5b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:20.845456 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7c784-6860-4a45-81f1-195ce85a5b94 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:20.846136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7c784-6860-4a45-81f1-195ce85a5b94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:20.846559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7c784-6860-4a45-81f1-195ce85a5b94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:20.847960 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7c784-6860-4a45-81f1-195ce85a5b94 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:20.874106 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 277/549] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:20.881963 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7c784-6860-4a45-81f1-195ce85a5b94 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:20.962327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362172b5-7583-42aa-92c4-7a35f9fb14de None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:21.004925 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:21.024221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2505f-ddb6-408e-a2b1-d75bdd6668e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:21.024928 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2505f-ddb6-408e-a2b1-d75bdd6668e2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:21.025211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2505f-ddb6-408e-a2b1-d75bdd6668e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:21.025482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2505f-ddb6-408e-a2b1-d75bdd6668e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:21.042894 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2505f-ddb6-408e-a2b1-d75bdd6668e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-7df2505f-ddb6-408e-a2b1-d75bdd6668e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:21.054337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:21.055467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:21.056300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:21.056300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:21.056300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:21.056300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:21.056300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:21.056300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:21.056300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:21.056300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:21.056300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:21.056300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:21.056300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:21.056300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:21.063059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 273/550] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:21.067771 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2505f-ddb6-408e-a2b1-d75bdd6668e2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:21.083601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7c784-6860-4a45-81f1-195ce85a5b94 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:21.105464 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:21.174021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695c167-8bcb-4571-af78-a5e29a9873fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:21.174693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695c167-8bcb-4571-af78-a5e29a9873fe None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:21.174971 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695c167-8bcb-4571-af78-a5e29a9873fe None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:21.175241 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695c167-8bcb-4571-af78-a5e29a9873fe None admin] PATH_INFO: `/v3/users/swift` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:21.185520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695c167-8bcb-4571-af78-a5e29a9873fe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:21.187056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695c167-8bcb-4571-af78-a5e29a9873fe None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-6695c167-8bcb-4571-af78-a5e29a9873fe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:21.206157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:21.207162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:21.208254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:21.208254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:21.208254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:21.208254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:21.208254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:21.208254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:21.208254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:21.208254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:21.208254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:53:21.208254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:21.208254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 278/551] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:53:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:21.209424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695c167-8bcb-4571-af78-a5e29a9873fe None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:21.245873 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2505f-ddb6-408e-a2b1-d75bdd6668e2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:21.322950 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11bce99-811d-42da-a3f0-9b5dcc1c9aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:21.323494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bce99-811d-42da-a3f0-9b5dcc1c9aac None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:21.323772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bce99-811d-42da-a3f0-9b5dcc1c9aac None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:21.324041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bce99-811d-42da-a3f0-9b5dcc1c9aac None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:21.324717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11bce99-811d-42da-a3f0-9b5dcc1c9aac None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:21.325017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11bce99-811d-42da-a3f0-9b5dcc1c9aac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:21.326587 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11bce99-811d-42da-a3f0-9b5dcc1c9aac None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:21.342254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 274/552] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:21.347531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11bce99-811d-42da-a3f0-9b5dcc1c9aac None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:21.349120 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695c167-8bcb-4571-af78-a5e29a9873fe None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:21.404244 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:21.432947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a36c45-d715-4f15-9a16-3b8e4d5551a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:21.433563 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a36c45-d715-4f15-9a16-3b8e4d5551a7 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:21.433842 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a36c45-d715-4f15-9a16-3b8e4d5551a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:21.434178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a36c45-d715-4f15-9a16-3b8e4d5551a7 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:21.435511 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a36c45-d715-4f15-9a16-3b8e4d5551a7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:21.435865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a36c45-d715-4f15-9a16-3b8e4d5551a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:21.437764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a36c45-d715-4f15-9a16-3b8e4d5551a7 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:21.476464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 279/553] 10.222.0.62 () {56 vars in 1135 bytes} [Fri Sep 9 11:53:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:21.482098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a36c45-d715-4f15-9a16-3b8e4d5551a7 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:21.487442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:21.511699 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11bce99-811d-42da-a3f0-9b5dcc1c9aac None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:21.560578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a442a-5dfa-408c-b685-2d686f0c987f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:21.561832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a442a-5dfa-408c-b685-2d686f0c987f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:21.562198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a442a-5dfa-408c-b685-2d686f0c987f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:21.562958 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a442a-5dfa-408c-b685-2d686f0c987f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:21.575714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a442a-5dfa-408c-b685-2d686f0c987f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:21.577096 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a442a-5dfa-408c-b685-2d686f0c987f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-b46a442a-5dfa-408c-b685-2d686f0c987f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:21.595153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:21.596144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:21.597030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:21.597030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:21.597030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:21.597198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 275/554] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:21.601881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0451b7-ce95-4f73-bf56-b1808b6ccee0 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:21.601881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0451b7-ce95-4f73-bf56-b1808b6ccee0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:21.601881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0451b7-ce95-4f73-bf56-b1808b6ccee0 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:21.601881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 276/555] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:21.605328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0451b7-ce95-4f73-bf56-b1808b6ccee0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:21.623429 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a36c45-d715-4f15-9a16-3b8e4d5551a7 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:21.672228 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc93fc8-9862-47ed-94e4-e37e5b10fec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:21.673129 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc93fc8-9862-47ed-94e4-e37e5b10fec5 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:21.673419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc93fc8-9862-47ed-94e4-e37e5b10fec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:21.674146 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc93fc8-9862-47ed-94e4-e37e5b10fec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:21.723216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0451b7-ce95-4f73-bf56-b1808b6ccee0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:21.747273 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:21.760096 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:21.766314 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e84d9-784c-4e18-b4cf-3d38da99f05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:21.767151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e84d9-784c-4e18-b4cf-3d38da99f05b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:21.767529 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e84d9-784c-4e18-b4cf-3d38da99f05b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:21.767993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e84d9-784c-4e18-b4cf-3d38da99f05b None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:21.768738 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e84d9-784c-4e18-b4cf-3d38da99f05b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:21.769139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e84d9-784c-4e18-b4cf-3d38da99f05b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:21.770709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e84d9-784c-4e18-b4cf-3d38da99f05b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:21.775154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc93fc8-9862-47ed-94e4-e37e5b10fec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:21.776593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc93fc8-9862-47ed-94e4-e37e5b10fec5 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:21.784690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 277/556] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:21.786170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418d758-179c-42f9-b8bf-f5cf256e8645 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:21.786612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418d758-179c-42f9-b8bf-f5cf256e8645 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:21.786978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418d758-179c-42f9-b8bf-f5cf256e8645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:21.807664 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a418d758-179c-42f9-b8bf-f5cf256e8645 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:21.809193 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a418d758-179c-42f9-b8bf-f5cf256e8645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:21.898963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 278/557] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:21.899900 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418d758-179c-42f9-b8bf-f5cf256e8645 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:21.926872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 280/558] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:53:21.932761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc93fc8-9862-47ed-94e4-e37e5b10fec5 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:22.026422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418d758-179c-42f9-b8bf-f5cf256e8645 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:22.071847 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:22.098354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a0421-af75-4013-bdd2-c2a4e3394866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:22.098704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:22.099000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a0421-af75-4013-bdd2-c2a4e3394866 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:22.099276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a0421-af75-4013-bdd2-c2a4e3394866 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:22.099635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a0421-af75-4013-bdd2-c2a4e3394866 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:22.111449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a0421-af75-4013-bdd2-c2a4e3394866 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:22.113829 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a0421-af75-4013-bdd2-c2a4e3394866 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:22.121469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc93fc8-9862-47ed-94e4-e37e5b10fec5 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-a81a0421-af75-4013-bdd2-c2a4e3394866 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:22.128016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:22.128962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:22.129907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:22.129907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:22.129907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:22.129907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:22.129907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:22.129907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:22.129907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:22.129907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:22.129907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:53:22.129907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:22.133204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 279/559] 10.222.0.62 () {56 vars in 1093 bytes} [Fri Sep 9 11:53:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:22.134793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0667ee2-a538-44ac-9b23-d54c25755331 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:22.135257 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0667ee2-a538-44ac-9b23-d54c25755331 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:22.135705 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0667ee2-a538-44ac-9b23-d54c25755331 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:22.142238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 280/560] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:22.143061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0667ee2-a538-44ac-9b23-d54c25755331 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:22.171731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fbd76-4002-4cf4-b341-ec77f6675c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:22.172292 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fbd76-4002-4cf4-b341-ec77f6675c99 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:22.172619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fbd76-4002-4cf4-b341-ec77f6675c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:22.172893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fbd76-4002-4cf4-b341-ec77f6675c99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:22.189100 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fbd76-4002-4cf4-b341-ec77f6675c99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:22.190612 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fbd76-4002-4cf4-b341-ec77f6675c99 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-7b7fbd76-4002-4cf4-b341-ec77f6675c99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:22.210831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:22.211931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:22.212823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:22.212823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:22.212823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:22.212823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 281/561] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:22.219646 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cde4ee-fb5d-442b-8346-18831dd3d1cf None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:22.220025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cde4ee-fb5d-442b-8346-18831dd3d1cf None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:22.220188 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cde4ee-fb5d-442b-8346-18831dd3d1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:22.242730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-17cde4ee-fb5d-442b-8346-18831dd3d1cf None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:22.243973 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-17cde4ee-fb5d-442b-8346-18831dd3d1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:22.288077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0667ee2-a538-44ac-9b23-d54c25755331 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:22.316235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 282/562] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:22.317305 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cde4ee-fb5d-442b-8346-18831dd3d1cf None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:22.330554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a260b-7b6b-403c-b9ca-0443aa22d9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:22.331503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a260b-7b6b-403c-b9ca-0443aa22d9ef None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:22.331959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a260b-7b6b-403c-b9ca-0443aa22d9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:22.332637 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a260b-7b6b-403c-b9ca-0443aa22d9ef None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:22.334041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a260b-7b6b-403c-b9ca-0443aa22d9ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:22.334524 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a260b-7b6b-403c-b9ca-0443aa22d9ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:22.336945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a260b-7b6b-403c-b9ca-0443aa22d9ef None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:22.361935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 281/563] 10.222.0.62 () {56 vars in 1133 bytes} [Fri Sep 9 11:53:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:22.363775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a260b-7b6b-403c-b9ca-0443aa22d9ef None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:22.387382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:22.411980 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:22.429766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cde4ee-fb5d-442b-8346-18831dd3d1cf None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:22.466947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a02a8-7d4a-4877-bec7-89e89bc344ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:22.467549 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a02a8-7d4a-4877-bec7-89e89bc344ec None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:22.467828 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a02a8-7d4a-4877-bec7-89e89bc344ec None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:22.468099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a02a8-7d4a-4877-bec7-89e89bc344ec None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:22.468913 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a02a8-7d4a-4877-bec7-89e89bc344ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:22.469356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a02a8-7d4a-4877-bec7-89e89bc344ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:22.470754 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a02a8-7d4a-4877-bec7-89e89bc344ec None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:22.471937 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a260b-7b6b-403c-b9ca-0443aa22d9ef None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:22.487485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 283/564] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:22.487485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809dd7a-7492-48bd-af00-4c361ab5f696 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:22.487485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809dd7a-7492-48bd-af00-4c361ab5f696 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:22.487485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809dd7a-7492-48bd-af00-4c361ab5f696 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:22.487485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 284/565] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:22.487485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809dd7a-7492-48bd-af00-4c361ab5f696 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:22.516009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf82cdb-7f39-4a94-88ab-aac145d2a0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:22.516643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf82cdb-7f39-4a94-88ab-aac145d2a0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:22.516920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf82cdb-7f39-4a94-88ab-aac145d2a0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:22.517189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf82cdb-7f39-4a94-88ab-aac145d2a0e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:22.527648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf82cdb-7f39-4a94-88ab-aac145d2a0e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-bcf82cdb-7f39-4a94-88ab-aac145d2a0e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:22.532439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:22.533388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:22.534263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:22.534263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:22.534263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:22.534263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:22.534263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:22.534263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:22.534263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:22.534263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:22.534263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:22.534263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:22.534263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:22.534263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:22.534263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 282/566] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:22.534971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf82cdb-7f39-4a94-88ab-aac145d2a0e2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:22.601580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809dd7a-7492-48bd-af00-4c361ab5f696 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:22.625222 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:22.635816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa40fe3-2dcf-4ac2-b433-e0d3cfd55710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:22.636441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa40fe3-2dcf-4ac2-b433-e0d3cfd55710 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:22.636722 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa40fe3-2dcf-4ac2-b433-e0d3cfd55710 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:22.637004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa40fe3-2dcf-4ac2-b433-e0d3cfd55710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:22.643676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:22.646353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf82cdb-7f39-4a94-88ab-aac145d2a0e2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:22.647761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa40fe3-2dcf-4ac2-b433-e0d3cfd55710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:22.649394 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa40fe3-2dcf-4ac2-b433-e0d3cfd55710 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-eaa40fe3-2dcf-4ac2-b433-e0d3cfd55710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:22.664395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:22.665365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:22.666287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:22.666287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:22.666287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:22.666287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 285/567] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:22.666782 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc307a-aa97-41c5-a29f-6e1721a616a5 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:22.667040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc307a-aa97-41c5-a29f-6e1721a616a5 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:22.667316 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc307a-aa97-41c5-a29f-6e1721a616a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:22.683557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66fe9c-c52c-442d-8701-d78d9d142c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:22.684190 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66fe9c-c52c-442d-8701-d78d9d142c6b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:22.684499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66fe9c-c52c-442d-8701-d78d9d142c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:22.684801 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66fe9c-c52c-442d-8701-d78d9d142c6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:22.692221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-e7bc307a-aa97-41c5-a29f-6e1721a616a5 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:22.692400 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66fe9c-c52c-442d-8701-d78d9d142c6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:22.693527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e7bc307a-aa97-41c5-a29f-6e1721a616a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:22.693995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66fe9c-c52c-442d-8701-d78d9d142c6b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:22.705230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-de66fe9c-c52c-442d-8701-d78d9d142c6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:22.705230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:22.705230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:22.705230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:22.705230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:22.705230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:22.705230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:22.705230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:22.705230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:22.705230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:22.705230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:22.705230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:22.705230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:22.705230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:22.705230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:22.706379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:22.707497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:22.707497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:22.707497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:22.707497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:22.707497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:22.707497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:22.707497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:22.707497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:22.707497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 283/568] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:22.707497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66fe9c-c52c-442d-8701-d78d9d142c6b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:22.767219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 286/569] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:22.768102 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc307a-aa97-41c5-a29f-6e1721a616a5 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:22.819761 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66fe9c-c52c-442d-8701-d78d9d142c6b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:22.857809 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc01ef-8cac-4b0b-9fb4-691042ec33d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:22.858464 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc01ef-8cac-4b0b-9fb4-691042ec33d5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:22.858993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc01ef-8cac-4b0b-9fb4-691042ec33d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:22.859267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc01ef-8cac-4b0b-9fb4-691042ec33d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:22.859926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc01ef-8cac-4b0b-9fb4-691042ec33d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:22.860236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc01ef-8cac-4b0b-9fb4-691042ec33d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:22.861593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc01ef-8cac-4b0b-9fb4-691042ec33d5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:22.875069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 284/570] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:22.875069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc01ef-8cac-4b0b-9fb4-691042ec33d5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:22.890807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc307a-aa97-41c5-a29f-6e1721a616a5 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:22.899048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:22.904244 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:22.928682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec8446-db4d-4661-a7b5-f3bb1c6ce3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:22.929486 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec8446-db4d-4661-a7b5-f3bb1c6ce3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:22.929841 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec8446-db4d-4661-a7b5-f3bb1c6ce3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:22.930148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec8446-db4d-4661-a7b5-f3bb1c6ce3c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:22.941806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec8446-db4d-4661-a7b5-f3bb1c6ce3c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-bfec8446-db4d-4661-a7b5-f3bb1c6ce3c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:22.946726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:22.947849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:22.948870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:22.948870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:22.948870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:22.948870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:22.948870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:22.948870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:22.948870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:22.948870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:22.948870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:22.948870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:22.948870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:22.948870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:22.948870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 287/571] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:22.948870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec8446-db4d-4661-a7b5-f3bb1c6ce3c4 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:22.997727 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc01ef-8cac-4b0b-9fb4-691042ec33d5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:23.034128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe533a-230a-410c-8539-0939afa8198e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:23.034970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe533a-230a-410c-8539-0939afa8198e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:23.035113 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe533a-230a-410c-8539-0939afa8198e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:23.035457 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe533a-230a-410c-8539-0939afa8198e None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:23.035960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe533a-230a-410c-8539-0939afa8198e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:23.036260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe533a-230a-410c-8539-0939afa8198e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:23.037638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe533a-230a-410c-8539-0939afa8198e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:23.051392 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 285/572] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:23.052223 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe533a-230a-410c-8539-0939afa8198e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:23.062011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec8446-db4d-4661-a7b5-f3bb1c6ce3c4 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:23.096257 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fef33-962e-456f-9bf1-2c6111e5dc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:23.096858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fef33-962e-456f-9bf1-2c6111e5dc4c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:23.097133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fef33-962e-456f-9bf1-2c6111e5dc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:23.097408 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fef33-962e-456f-9bf1-2c6111e5dc4c None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:23.098093 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fef33-962e-456f-9bf1-2c6111e5dc4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:23.098420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fef33-962e-456f-9bf1-2c6111e5dc4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:23.099856 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fef33-962e-456f-9bf1-2c6111e5dc4c None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:23.112321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 288/573] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:23.113148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fef33-962e-456f-9bf1-2c6111e5dc4c None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:23.138683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:23.143713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:23.157497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe533a-230a-410c-8539-0939afa8198e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:23.194648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915da0f-76e8-4cc1-a7be-804ddb4231d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:23.194648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915da0f-76e8-4cc1-a7be-804ddb4231d3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:23.194648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915da0f-76e8-4cc1-a7be-804ddb4231d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:23.194648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915da0f-76e8-4cc1-a7be-804ddb4231d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:23.203851 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915da0f-76e8-4cc1-a7be-804ddb4231d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:23.205295 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915da0f-76e8-4cc1-a7be-804ddb4231d3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-a915da0f-76e8-4cc1-a7be-804ddb4231d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:23.221195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:23.222214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:23.223116 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:23.223116 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:23.223116 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:23.223116 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 286/574] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:23.223116 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915da0f-76e8-4cc1-a7be-804ddb4231d3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:23.233191 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fef33-962e-456f-9bf1-2c6111e5dc4c None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:23.268905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d306a3f-fa28-45f8-86fc-16dfa4ada28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:23.269556 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d306a3f-fa28-45f8-86fc-16dfa4ada28b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:23.269834 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d306a3f-fa28-45f8-86fc-16dfa4ada28b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:23.270096 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d306a3f-fa28-45f8-86fc-16dfa4ada28b None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:23.270913 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d306a3f-fa28-45f8-86fc-16dfa4ada28b None admin] RBAC: Adding query filter params (name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:23.271212 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d306a3f-fa28-45f8-86fc-16dfa4ada28b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:23.272565 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d306a3f-fa28-45f8-86fc-16dfa4ada28b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:23.280940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 289/575] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:23.281715 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d306a3f-fa28-45f8-86fc-16dfa4ada28b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:23.326843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915da0f-76e8-4cc1-a7be-804ddb4231d3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:23.359886 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733399bb-8563-4e6a-8633-416fdb9455b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:23.360500 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733399bb-8563-4e6a-8633-416fdb9455b6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:23.360771 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733399bb-8563-4e6a-8633-416fdb9455b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:23.361037 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733399bb-8563-4e6a-8633-416fdb9455b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:23.362849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:23.369807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:23.372437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733399bb-8563-4e6a-8633-416fdb9455b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-733399bb-8563-4e6a-8633-416fdb9455b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:23.377627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:23.378673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:23.379488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:23.379488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:23.379488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:23.379488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:23.379488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:23.379488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:23.379488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:23.379488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:23.379488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:23.379488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:23.379488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:23.379488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:23.379488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 287/576] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:23.380167 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733399bb-8563-4e6a-8633-416fdb9455b6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:23.385705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d306a3f-fa28-45f8-86fc-16dfa4ada28b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:23.419950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820418d-f48c-4e68-ba0b-036bfc4e8d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:23.420589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820418d-f48c-4e68-ba0b-036bfc4e8d54 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:23.420843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820418d-f48c-4e68-ba0b-036bfc4e8d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:23.421116 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820418d-f48c-4e68-ba0b-036bfc4e8d54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:23.428530 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820418d-f48c-4e68-ba0b-036bfc4e8d54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:23.430133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820418d-f48c-4e68-ba0b-036bfc4e8d54 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:23.441420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-5820418d-f48c-4e68-ba0b-036bfc4e8d54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:23.441420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:23.441420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:23.441420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:23.441420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:23.441420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:23.441420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:23.441420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:23.441420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:23.441420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:23.441420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:23.441420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:23.441420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:23.441420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:23.441420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:23.442606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:23.443500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:23.443500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:23.443500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:23.443500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:23.443500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:23.443500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:23.443500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:23.443500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:23.443500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 290/577] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:23.443500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820418d-f48c-4e68-ba0b-036bfc4e8d54 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:23.488654 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733399bb-8563-4e6a-8633-416fdb9455b6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:23.521945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89fd86-8e7b-4ee8-9bfb-5bbf0d974a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:23.522781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89fd86-8e7b-4ee8-9bfb-5bbf0d974a05 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:23.523519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89fd86-8e7b-4ee8-9bfb-5bbf0d974a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:23.523792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89fd86-8e7b-4ee8-9bfb-5bbf0d974a05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:23.524515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89fd86-8e7b-4ee8-9bfb-5bbf0d974a05 None admin] RBAC: Adding query filter params (role.id=035f7d9f445d406eb2e83ca90fc4d5ca, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=1b2de52919e54bf9bcc352909c9f905f) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:23.524829 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89fd86-8e7b-4ee8-9bfb-5bbf0d974a05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:23.526287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89fd86-8e7b-4ee8-9bfb-5bbf0d974a05 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:23.532291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 288/578] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:23 2022] GET /identity/v3/role_assignments?user.id=1b2de52919e54bf9bcc352909c9f905f&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=035f7d9f445d406eb2e83ca90fc4d5ca => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:23.533113 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89fd86-8e7b-4ee8-9bfb-5bbf0d974a05 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:23.548302 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820418d-f48c-4e68-ba0b-036bfc4e8d54 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:23.585651 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5818d-1ebd-4c99-ab9a-f17db9cf24ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:23.586302 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5818d-1ebd-4c99-ab9a-f17db9cf24ad None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:23.586579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5818d-1ebd-4c99-ab9a-f17db9cf24ad None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:23.586844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5818d-1ebd-4c99-ab9a-f17db9cf24ad None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:23.587505 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5818d-1ebd-4c99-ab9a-f17db9cf24ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:23.587806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5818d-1ebd-4c99-ab9a-f17db9cf24ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:23.589215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5818d-1ebd-4c99-ab9a-f17db9cf24ad None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:23.603284 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 291/579] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:23.604148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5818d-1ebd-4c99-ab9a-f17db9cf24ad None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:23.609177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:23.618101 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:23.667247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89fd86-8e7b-4ee8-9bfb-5bbf0d974a05 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:23.708681 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f426172-899c-4195-9888-3838962cc76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:23.709319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f426172-899c-4195-9888-3838962cc76d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:23.709596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f426172-899c-4195-9888-3838962cc76d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:23.709861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f426172-899c-4195-9888-3838962cc76d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:23.723068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f426172-899c-4195-9888-3838962cc76d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:23.724677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f426172-899c-4195-9888-3838962cc76d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:23.724830 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5818d-1ebd-4c99-ab9a-f17db9cf24ad None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-6f426172-899c-4195-9888-3838962cc76d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:23.743303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:23.744288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:23.745198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:23.745198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:23.745198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:23.745198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 289/580] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:23.745198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f426172-899c-4195-9888-3838962cc76d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:23.770487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42381c8-05eb-4b5a-9682-99c733e43018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:23.771443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42381c8-05eb-4b5a-9682-99c733e43018 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:23.771954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42381c8-05eb-4b5a-9682-99c733e43018 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:23.772437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42381c8-05eb-4b5a-9682-99c733e43018 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:23.773249 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42381c8-05eb-4b5a-9682-99c733e43018 None admin] RBAC: Adding query filter params (name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:23.773663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42381c8-05eb-4b5a-9682-99c733e43018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:23.775104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42381c8-05eb-4b5a-9682-99c733e43018 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:23.783852 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 292/581] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:23.789117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42381c8-05eb-4b5a-9682-99c733e43018 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:23.853991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f426172-899c-4195-9888-3838962cc76d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:23.869183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:23.872666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:23.900518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38aa55-e086-4cee-877e-82eb7db93f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:23.901127 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38aa55-e086-4cee-877e-82eb7db93f25 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:23.901399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38aa55-e086-4cee-877e-82eb7db93f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:23.901665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38aa55-e086-4cee-877e-82eb7db93f25 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:23.902534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38aa55-e086-4cee-877e-82eb7db93f25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:23.903170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38aa55-e086-4cee-877e-82eb7db93f25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:23.904704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38aa55-e086-4cee-877e-82eb7db93f25 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:23.908717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42381c8-05eb-4b5a-9682-99c733e43018 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:23.921792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 290/582] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:23.921792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38aa55-e086-4cee-877e-82eb7db93f25 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:23.952588 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36baccf-1b68-4858-a24b-28e9a7869e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:23.953346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36baccf-1b68-4858-a24b-28e9a7869e39 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:23.953696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36baccf-1b68-4858-a24b-28e9a7869e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:23.953975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36baccf-1b68-4858-a24b-28e9a7869e39 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:23.963680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36baccf-1b68-4858-a24b-28e9a7869e39 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:23.965128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36baccf-1b68-4858-a24b-28e9a7869e39 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-e36baccf-1b68-4858-a24b-28e9a7869e39 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:23.976919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:23.983198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:23.984573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:23.984573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:23.984573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:23.984573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:23.984573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:23.984573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:23.984573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:23.984573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:23.984573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 11:53:23.984573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:23.984573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 293/583] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:23.984573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36baccf-1b68-4858-a24b-28e9a7869e39 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:24.038342 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38aa55-e086-4cee-877e-82eb7db93f25 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:24.076484 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c6f22-4078-45d3-9b6c-6f71155f1b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:24.077099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c6f22-4078-45d3-9b6c-6f71155f1b03 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:24.077375 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c6f22-4078-45d3-9b6c-6f71155f1b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:24.077647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c6f22-4078-45d3-9b6c-6f71155f1b03 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:24.078330 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c6f22-4078-45d3-9b6c-6f71155f1b03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:24.078633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c6f22-4078-45d3-9b6c-6f71155f1b03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:24.079984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c6f22-4078-45d3-9b6c-6f71155f1b03 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:24.086382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36baccf-1b68-4858-a24b-28e9a7869e39 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:24.096823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 291/584] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:24.096823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c6f22-4078-45d3-9b6c-6f71155f1b03 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:24.120741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:24.127622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa43fc-52b5-4f80-8ce1-dd4ace5e13ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:24.128358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa43fc-52b5-4f80-8ce1-dd4ace5e13ea None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:24.128707 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa43fc-52b5-4f80-8ce1-dd4ace5e13ea None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:24.129065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa43fc-52b5-4f80-8ce1-dd4ace5e13ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:24.130992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:24.141925 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa43fc-52b5-4f80-8ce1-dd4ace5e13ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:24.143504 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa43fc-52b5-4f80-8ce1-dd4ace5e13ea None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-c5fa43fc-52b5-4f80-8ce1-dd4ace5e13ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:24.158490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:24.159526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:24.160460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:24.160460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:24.160460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:24.160460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 294/585] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:24.161296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa43fc-52b5-4f80-8ce1-dd4ace5e13ea None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:24.217243 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c6f22-4078-45d3-9b6c-6f71155f1b03 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:24.258798 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d3d81-1525-403f-ac78-73ad09de72c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:24.259546 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d3d81-1525-403f-ac78-73ad09de72c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:24.259767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d3d81-1525-403f-ac78-73ad09de72c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:24.260052 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d3d81-1525-403f-ac78-73ad09de72c6 None admin] PATH_INFO: `/v3/projects/32b06df0c5504592b012585f193386e7/users/1b30e80c421245b5a1ed5849b28ea90e/roles/035f7d9f445d406eb2e83ca90fc4d5ca` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:24.273511 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa43fc-52b5-4f80-8ce1-dd4ace5e13ea None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:24.279621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d3d81-1525-403f-ac78-73ad09de72c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=32b06df0c5504592b012585f193386e7, user_id=1b30e80c421245b5a1ed5849b28ea90e, role_id=035f7d9f445d406eb2e83ca90fc4d5ca)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:24.281064 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d3d81-1525-403f-ac78-73ad09de72c6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:24.300438 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 292/586] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:53:24 2022] PUT /identity/v3/projects/32b06df0c5504592b012585f193386e7/users/1b30e80c421245b5a1ed5849b28ea90e/roles/035f7d9f445d406eb2e83ca90fc4d5ca => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:24.302394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d3d81-1525-403f-ac78-73ad09de72c6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:24.324756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827f7b9-b5c0-490f-820e-b7f29a2488a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:24.325794 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827f7b9-b5c0-490f-820e-b7f29a2488a8 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:24.326336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827f7b9-b5c0-490f-820e-b7f29a2488a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:24.326907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827f7b9-b5c0-490f-820e-b7f29a2488a8 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:24.327931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827f7b9-b5c0-490f-820e-b7f29a2488a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:24.328443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827f7b9-b5c0-490f-820e-b7f29a2488a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:24.330345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827f7b9-b5c0-490f-820e-b7f29a2488a8 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:24.354316 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 295/587] 10.222.0.62 () {56 vars in 1137 bytes} [Fri Sep 9 11:53:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:24.355607 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827f7b9-b5c0-490f-820e-b7f29a2488a8 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:24.413070 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:24.434074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:24.460463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d3d81-1525-403f-ac78-73ad09de72c6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:24.486265 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827f7b9-b5c0-490f-820e-b7f29a2488a8 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:24.506154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dec43-4504-404e-9e88-d25ae7d85ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:24.506925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dec43-4504-404e-9e88-d25ae7d85ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:24.508163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dec43-4504-404e-9e88-d25ae7d85ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:24.508515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dec43-4504-404e-9e88-d25ae7d85ba3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:24.517338 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707dec43-4504-404e-9e88-d25ae7d85ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:24.519415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707dec43-4504-404e-9e88-d25ae7d85ba3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:24.528319 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a442ea2-9c84-4376-8936-e1e48fe6d47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:24.529011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a442ea2-9c84-4376-8936-e1e48fe6d47d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:24.529358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a442ea2-9c84-4376-8936-e1e48fe6d47d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:24.529697 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a442ea2-9c84-4376-8936-e1e48fe6d47d None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:24.530490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a442ea2-9c84-4376-8936-e1e48fe6d47d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:24.530815 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a442ea2-9c84-4376-8936-e1e48fe6d47d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:24.532320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a442ea2-9c84-4376-8936-e1e48fe6d47d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-707dec43-4504-404e-9e88-d25ae7d85ba3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:24.532827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:24.534076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:24.535118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:24.535118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:24.535118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:24.535118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:24.535118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:24.535118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:24.535118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:24.535118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:24.535118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:53:24.535118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:24.535118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 293/588] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:24.535861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dec43-4504-404e-9e88-d25ae7d85ba3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:24.548917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 296/589] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:24.550007 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a442ea2-9c84-4376-8936-e1e48fe6d47d None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:24.655339 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:24.660773 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a442ea2-9c84-4376-8936-e1e48fe6d47d None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:24.662378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dec43-4504-404e-9e88-d25ae7d85ba3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:24.701518 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b22bf6-cd4f-4e2e-99bc-f300d4ec8664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:24.702460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b22bf6-cd4f-4e2e-99bc-f300d4ec8664 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:24.702683 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b22bf6-cd4f-4e2e-99bc-f300d4ec8664 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:24.703077 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b22bf6-cd4f-4e2e-99bc-f300d4ec8664 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:24.703931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b22bf6-cd4f-4e2e-99bc-f300d4ec8664 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:24.704380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b22bf6-cd4f-4e2e-99bc-f300d4ec8664 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:24.706110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b22bf6-cd4f-4e2e-99bc-f300d4ec8664 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:24.708561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841c538-9f7b-4aca-83ab-5a2dff7aa072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:24.711037 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841c538-9f7b-4aca-83ab-5a2dff7aa072 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:24.711494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841c538-9f7b-4aca-83ab-5a2dff7aa072 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:24.711922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841c538-9f7b-4aca-83ab-5a2dff7aa072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:24.724657 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:24.730792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841c538-9f7b-4aca-83ab-5a2dff7aa072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:24.731644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 297/590] 10.222.0.62 () {56 vars in 1137 bytes} [Fri Sep 9 11:53:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:24.732625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b22bf6-cd4f-4e2e-99bc-f300d4ec8664 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:24.732809 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841c538-9f7b-4aca-83ab-5a2dff7aa072 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-d841c538-9f7b-4aca-83ab-5a2dff7aa072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:24.756004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:24.757282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:24.758439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:24.758439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:24.758439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:24.764076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 294/591] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:24.765096 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841c538-9f7b-4aca-83ab-5a2dff7aa072 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:24.868818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b22bf6-cd4f-4e2e-99bc-f300d4ec8664 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:24.902878 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841c538-9f7b-4aca-83ab-5a2dff7aa072 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:24.907505 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de658a2a-474f-414f-8a7c-1f2ea44946bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:24.908148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de658a2a-474f-414f-8a7c-1f2ea44946bc None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:24.908494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de658a2a-474f-414f-8a7c-1f2ea44946bc None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:24.908831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de658a2a-474f-414f-8a7c-1f2ea44946bc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:24.916851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de658a2a-474f-414f-8a7c-1f2ea44946bc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:24.918423 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de658a2a-474f-414f-8a7c-1f2ea44946bc None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-de658a2a-474f-414f-8a7c-1f2ea44946bc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:24.931482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:24.932674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:24.933847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:24.933847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:24.933847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:24.933847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:24.933847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:24.933847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:24.933847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:24.933847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:24.933847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 11:53:24.933847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:24.934714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 298/592] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:24.935499 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de658a2a-474f-414f-8a7c-1f2ea44946bc None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:24.946507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34283790-fd24-41ca-8808-75e494b07ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:24.947419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34283790-fd24-41ca-8808-75e494b07ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:24.947880 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34283790-fd24-41ca-8808-75e494b07ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:24.948347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34283790-fd24-41ca-8808-75e494b07ce4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:24.956932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:24.963874 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34283790-fd24-41ca-8808-75e494b07ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:24.966471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34283790-fd24-41ca-8808-75e494b07ce4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-34283790-fd24-41ca-8808-75e494b07ce4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:24.983031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:24.984478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:24.985479 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:24.985479 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:24.985479 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:24.987862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 295/593] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:24.988795 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34283790-fd24-41ca-8808-75e494b07ce4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:25.050889 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:25.053978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de658a2a-474f-414f-8a7c-1f2ea44946bc None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:25.096761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ed28f-eb93-427f-9ccc-d86281439d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:25.097558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ed28f-eb93-427f-9ccc-d86281439d7a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:25.097794 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ed28f-eb93-427f-9ccc-d86281439d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:25.098068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ed28f-eb93-427f-9ccc-d86281439d7a None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:25.098756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ed28f-eb93-427f-9ccc-d86281439d7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:25.099561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ed28f-eb93-427f-9ccc-d86281439d7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:25.100791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ed28f-eb93-427f-9ccc-d86281439d7a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:25.114847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 299/594] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:25.115715 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ed28f-eb93-427f-9ccc-d86281439d7a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:25.122752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34283790-fd24-41ca-8808-75e494b07ce4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:25.162804 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63049aff-d321-4300-b335-a195fd2686ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:25.163786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63049aff-d321-4300-b335-a195fd2686ed None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:25.164257 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63049aff-d321-4300-b335-a195fd2686ed None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:25.164721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63049aff-d321-4300-b335-a195fd2686ed None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:25.165643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63049aff-d321-4300-b335-a195fd2686ed None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:25.166145 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63049aff-d321-4300-b335-a195fd2686ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:25.167755 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63049aff-d321-4300-b335-a195fd2686ed None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:25.188838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 296/595] 10.222.0.62 () {56 vars in 1139 bytes} [Fri Sep 9 11:53:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:25.190300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63049aff-d321-4300-b335-a195fd2686ed None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:25.207088 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:25.233230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ed28f-eb93-427f-9ccc-d86281439d7a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:25.273281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274fc04-8643-4ebf-a2a6-4ea702371061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:25.274101 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274fc04-8643-4ebf-a2a6-4ea702371061 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:25.274510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274fc04-8643-4ebf-a2a6-4ea702371061 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:25.274897 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274fc04-8643-4ebf-a2a6-4ea702371061 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:25.275765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274fc04-8643-4ebf-a2a6-4ea702371061 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:25.276163 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274fc04-8643-4ebf-a2a6-4ea702371061 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:25.277739 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274fc04-8643-4ebf-a2a6-4ea702371061 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:25.294618 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 300/596] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:25.295646 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274fc04-8643-4ebf-a2a6-4ea702371061 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:25.314169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63049aff-d321-4300-b335-a195fd2686ed None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:25.316663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:25.363273 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99e4a2-70ec-4ef1-8eb2-1a5756deec4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:25.364640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99e4a2-70ec-4ef1-8eb2-1a5756deec4d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:25.365185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99e4a2-70ec-4ef1-8eb2-1a5756deec4d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:25.365615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99e4a2-70ec-4ef1-8eb2-1a5756deec4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:25.375469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99e4a2-70ec-4ef1-8eb2-1a5756deec4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:25.377397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99e4a2-70ec-4ef1-8eb2-1a5756deec4d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:25.393616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-de99e4a2-70ec-4ef1-8eb2-1a5756deec4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:25.393616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:25.393616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:25.393616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:25.393616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:25.393616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:25.393616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:25.393616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:25.393616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:25.393616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:25.393616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:25.393616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:25.393616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:25.393616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:25.393616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:25.394810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:25.395923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:25.395923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:25.395923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:25.395923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:25.395923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:25.395923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:25.395923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:25.395923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:25.398624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 297/597] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:25.399706 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99e4a2-70ec-4ef1-8eb2-1a5756deec4d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:25.417810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274fc04-8643-4ebf-a2a6-4ea702371061 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:25.456725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71972fe6-5603-44cd-89fb-b64573fe8c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:25.457520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71972fe6-5603-44cd-89fb-b64573fe8c0b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:25.457908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71972fe6-5603-44cd-89fb-b64573fe8c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:25.458307 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71972fe6-5603-44cd-89fb-b64573fe8c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:25.466571 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:25.472623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71972fe6-5603-44cd-89fb-b64573fe8c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:25.474206 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71972fe6-5603-44cd-89fb-b64573fe8c0b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-71972fe6-5603-44cd-89fb-b64573fe8c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:25.490439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:25.492636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:25.493765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:25.493765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:25.493765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:25.494836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 301/598] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:25.496468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71972fe6-5603-44cd-89fb-b64573fe8c0b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:25.533706 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99e4a2-70ec-4ef1-8eb2-1a5756deec4d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:25.573288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2a538-bd08-406a-b5d2-2331a642fe11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:25.574082 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2a538-bd08-406a-b5d2-2331a642fe11 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:25.574486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2a538-bd08-406a-b5d2-2331a642fe11 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:25.574852 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2a538-bd08-406a-b5d2-2331a642fe11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:25.582619 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2a538-bd08-406a-b5d2-2331a642fe11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:25.584288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2a538-bd08-406a-b5d2-2331a642fe11 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:25.596635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-28d2a538-bd08-406a-b5d2-2331a642fe11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:25.596635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:25.596635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:25.596635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:25.596635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:25.596635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:25.596635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:25.596635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:25.596635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:25.596635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:25.596635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:25.596635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:25.596635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:25.596635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:25.596635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:25.597793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:25.598931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:25.598931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:25.598931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:25.598931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:25.598931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:25.598931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:25.598931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:25.598931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:25.600480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 298/599] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:25.606561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2a538-bd08-406a-b5d2-2331a642fe11 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:25.630371 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71972fe6-5603-44cd-89fb-b64573fe8c0b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:25.635478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:25.683097 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d8ae9-69e8-4e90-9fe8-d5254d2943ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:25.683644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d8ae9-69e8-4e90-9fe8-d5254d2943ed None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:25.683923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d8ae9-69e8-4e90-9fe8-d5254d2943ed None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:25.684199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d8ae9-69e8-4e90-9fe8-d5254d2943ed None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:25.685426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d8ae9-69e8-4e90-9fe8-d5254d2943ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:25.685744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d8ae9-69e8-4e90-9fe8-d5254d2943ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:25.687573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d8ae9-69e8-4e90-9fe8-d5254d2943ed None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:25.708775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 302/600] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:25.709716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d8ae9-69e8-4e90-9fe8-d5254d2943ed None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:25.743609 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2a538-bd08-406a-b5d2-2331a642fe11 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:25.778521 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:25.785098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096e7b5-4c92-4b0e-b0c9-0c814b487b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:25.785754 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096e7b5-4c92-4b0e-b0c9-0c814b487b07 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:25.786073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096e7b5-4c92-4b0e-b0c9-0c814b487b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:25.786371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096e7b5-4c92-4b0e-b0c9-0c814b487b07 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:25.787064 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096e7b5-4c92-4b0e-b0c9-0c814b487b07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:25.787376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096e7b5-4c92-4b0e-b0c9-0c814b487b07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:25.789473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096e7b5-4c92-4b0e-b0c9-0c814b487b07 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:25.806212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 299/601] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:25.807110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096e7b5-4c92-4b0e-b0c9-0c814b487b07 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:25.842473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d8ae9-69e8-4e90-9fe8-d5254d2943ed None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:25.880944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf39409-d7e7-4251-9db5-37d85ed82450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:25.881539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf39409-d7e7-4251-9db5-37d85ed82450 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:25.881827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf39409-d7e7-4251-9db5-37d85ed82450 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:25.882154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf39409-d7e7-4251-9db5-37d85ed82450 None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:25.882866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf39409-d7e7-4251-9db5-37d85ed82450 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:25.883171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf39409-d7e7-4251-9db5-37d85ed82450 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:25.884628 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf39409-d7e7-4251-9db5-37d85ed82450 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:25.898172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 303/602] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:25.899434 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78a1cb-bfb0-40b6-a1ab-951a3b24e103 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:25.899719 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78a1cb-bfb0-40b6-a1ab-951a3b24e103 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:25.900042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78a1cb-bfb0-40b6-a1ab-951a3b24e103 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:25.901427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 304/603] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:25.902864 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78a1cb-bfb0-40b6-a1ab-951a3b24e103 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:25.905011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:25.925818 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096e7b5-4c92-4b0e-b0c9-0c814b487b07 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:25.963656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0154c3-71bf-4601-8f70-7f579547a8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:25.964776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0154c3-71bf-4601-8f70-7f579547a8af None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:25.965125 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0154c3-71bf-4601-8f70-7f579547a8af None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:25.965583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0154c3-71bf-4601-8f70-7f579547a8af None admin] PATH_INFO: `/v3/projects/32b06df0c5504592b012585f193386e7/users/2d477a49a1724b7e8a4699a5ebd04bde/roles/035f7d9f445d406eb2e83ca90fc4d5ca` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:25.981814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0154c3-71bf-4601-8f70-7f579547a8af None admin] RBAC: Authorizing `identity:create_grant(project_id=32b06df0c5504592b012585f193386e7, user_id=2d477a49a1724b7e8a4699a5ebd04bde, role_id=035f7d9f445d406eb2e83ca90fc4d5ca)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:25.984697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0154c3-71bf-4601-8f70-7f579547a8af None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:26.006201 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 300/604] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:53:25 2022] PUT /identity/v3/projects/32b06df0c5504592b012585f193386e7/users/2d477a49a1724b7e8a4699a5ebd04bde/roles/035f7d9f445d406eb2e83ca90fc4d5ca => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:26.006201 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0154c3-71bf-4601-8f70-7f579547a8af None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:26.028025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78a1cb-bfb0-40b6-a1ab-951a3b24e103 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:26.041680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:26.076579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f818a7c-4807-4683-865e-a0f4d1aa6784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:26.077927 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f818a7c-4807-4683-865e-a0f4d1aa6784 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:26.078494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f818a7c-4807-4683-865e-a0f4d1aa6784 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:26.078954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f818a7c-4807-4683-865e-a0f4d1aa6784 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:26.089322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f818a7c-4807-4683-865e-a0f4d1aa6784 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:26.091596 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f818a7c-4807-4683-865e-a0f4d1aa6784 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:26.105877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-1f818a7c-4807-4683-865e-a0f4d1aa6784 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:26.105877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:26.105877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:26.105877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:26.105877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:26.105877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:26.105877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:26.105877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:26.105877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:26.105877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:26.105877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:26.105877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:26.105877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:26.105877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:26.105877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:26.106846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:26.107728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:26.107728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:26.107728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:26.107728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:26.107728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:26.107728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:26.107728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:26.107728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:26.107728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 305/605] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:26.113737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a81691-cd36-4382-b2a7-0bc8da454aa5 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:26.114041 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a81691-cd36-4382-b2a7-0bc8da454aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:26.114316 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a81691-cd36-4382-b2a7-0bc8da454aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:26.134058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0154c3-71bf-4601-8f70-7f579547a8af None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:26.135659 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-e4a81691-cd36-4382-b2a7-0bc8da454aa5 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:26.136902 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e4a81691-cd36-4382-b2a7-0bc8da454aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:26.171311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:26.180410 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1521d56-61a8-4fcb-ae20-4cb9410b4b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:26.180673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1521d56-61a8-4fcb-ae20-4cb9410b4b59 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:26.180782 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1521d56-61a8-4fcb-ae20-4cb9410b4b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:26.181050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1521d56-61a8-4fcb-ae20-4cb9410b4b59 None admin] PATH_INFO: `/v3/projects/32b06df0c5504592b012585f193386e7/users/a25411d51a524f4b9f0ff846a0b29b6f/roles/035f7d9f445d406eb2e83ca90fc4d5ca` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:26.198211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1521d56-61a8-4fcb-ae20-4cb9410b4b59 None admin] RBAC: Authorizing `identity:create_grant(project_id=32b06df0c5504592b012585f193386e7, user_id=a25411d51a524f4b9f0ff846a0b29b6f, role_id=035f7d9f445d406eb2e83ca90fc4d5ca)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:26.200242 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1521d56-61a8-4fcb-ae20-4cb9410b4b59 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:26.220813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 301/606] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:53:25 2022] PUT /identity/v3/projects/32b06df0c5504592b012585f193386e7/users/a25411d51a524f4b9f0ff846a0b29b6f/roles/035f7d9f445d406eb2e83ca90fc4d5ca => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:26.221932 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1521d56-61a8-4fcb-ae20-4cb9410b4b59 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:26.246144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 306/607] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:26.348162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1521d56-61a8-4fcb-ae20-4cb9410b4b59 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:26.389424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb72fb-bcfc-4a95-9b07-0434070370b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:26.390050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb72fb-bcfc-4a95-9b07-0434070370b3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:26.390425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb72fb-bcfc-4a95-9b07-0434070370b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:26.390696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb72fb-bcfc-4a95-9b07-0434070370b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:26.391400 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb72fb-bcfc-4a95-9b07-0434070370b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:26.391701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb72fb-bcfc-4a95-9b07-0434070370b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:26.393221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb72fb-bcfc-4a95-9b07-0434070370b3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:26.407856 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 302/608] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:26.415289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a81691-cd36-4382-b2a7-0bc8da454aa5 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:26.551518 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:26.555689 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a81691-cd36-4382-b2a7-0bc8da454aa5 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:26.602581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26553f47-c995-433f-9802-034c5d36e225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:26.603247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26553f47-c995-433f-9802-034c5d36e225 None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:26.603521 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26553f47-c995-433f-9802-034c5d36e225 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:26.603787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26553f47-c995-433f-9802-034c5d36e225 None admin] PATH_INFO: `/v3/projects/32b06df0c5504592b012585f193386e7/users/d00634cdd3fb41e794d3af0b88ba7338/roles/035f7d9f445d406eb2e83ca90fc4d5ca` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:26.619780 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26553f47-c995-433f-9802-034c5d36e225 None admin] RBAC: Authorizing `identity:create_grant(project_id=32b06df0c5504592b012585f193386e7, user_id=d00634cdd3fb41e794d3af0b88ba7338, role_id=035f7d9f445d406eb2e83ca90fc4d5ca)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:26.621236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26553f47-c995-433f-9802-034c5d36e225 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:26.639143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 307/609] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:53:26 2022] PUT /identity/v3/projects/32b06df0c5504592b012585f193386e7/users/d00634cdd3fb41e794d3af0b88ba7338/roles/035f7d9f445d406eb2e83ca90fc4d5ca => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:26.647564 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb72fb-bcfc-4a95-9b07-0434070370b3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:26.690661 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65f78b-bba4-41cf-a0a2-9173b22896bb None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:26.690958 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65f78b-bba4-41cf-a0a2-9173b22896bb None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:26.691421 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65f78b-bba4-41cf-a0a2-9173b22896bb None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:26.694390 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 308/610] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:26.704614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:26.706325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717ed71-03b5-4672-b10d-dd21ef4c82f1 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:26.706745 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717ed71-03b5-4672-b10d-dd21ef4c82f1 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:26.707131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717ed71-03b5-4672-b10d-dd21ef4c82f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:26.726723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-0717ed71-03b5-4672-b10d-dd21ef4c82f1 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:26.728335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0717ed71-03b5-4672-b10d-dd21ef4c82f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:26.772401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb72fb-bcfc-4a95-9b07-0434070370b3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:26.812881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2615fbb-8ff0-4d64-9f62-2b2b23a9c522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:26.813436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2615fbb-8ff0-4d64-9f62-2b2b23a9c522 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:26.813714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2615fbb-8ff0-4d64-9f62-2b2b23a9c522 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:26.813985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2615fbb-8ff0-4d64-9f62-2b2b23a9c522 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:26.831551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2615fbb-8ff0-4d64-9f62-2b2b23a9c522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-e2615fbb-8ff0-4d64-9f62-2b2b23a9c522 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:26.843534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:26.844597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:26.845463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:26.845463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:26.845463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:26.845463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:26.845463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:26.845463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:26.845463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:26.845463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:26.845463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:26.845463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:26.845463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:26.845463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:26.845463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 303/611] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:26.852922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2615fbb-8ff0-4d64-9f62-2b2b23a9c522 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:26.880779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 309/612] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:26.978658 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2615fbb-8ff0-4d64-9f62-2b2b23a9c522 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:26.980180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:27.024457 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433b807-7d61-4b52-9783-8b1e5f01ee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:27.025075 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433b807-7d61-4b52-9783-8b1e5f01ee40 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:27.025345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433b807-7d61-4b52-9783-8b1e5f01ee40 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:27.025611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433b807-7d61-4b52-9783-8b1e5f01ee40 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:27.026329 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7433b807-7d61-4b52-9783-8b1e5f01ee40 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:27.026624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7433b807-7d61-4b52-9783-8b1e5f01ee40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:27.027993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7433b807-7d61-4b52-9783-8b1e5f01ee40 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:27.037003 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 304/613] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:27.043103 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717ed71-03b5-4672-b10d-dd21ef4c82f1 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:27.127123 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:27.149013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717ed71-03b5-4672-b10d-dd21ef4c82f1 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:27.207294 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75271387-0ff9-4894-a8e1-efac132748c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:27.207950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271387-0ff9-4894-a8e1-efac132748c6 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:27.208223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271387-0ff9-4894-a8e1-efac132748c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:27.208487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271387-0ff9-4894-a8e1-efac132748c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:27.227052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75271387-0ff9-4894-a8e1-efac132748c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:27.228678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75271387-0ff9-4894-a8e1-efac132748c6 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-75271387-0ff9-4894-a8e1-efac132748c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:27.245734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:27.246779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:27.247680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:27.247680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:27.247680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:27.247680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 310/614] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:27.252823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433b807-7d61-4b52-9783-8b1e5f01ee40 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:27.318730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75271387-0ff9-4894-a8e1-efac132748c6 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:27.394606 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433b807-7d61-4b52-9783-8b1e5f01ee40 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:27.453459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f478223-b9a8-4f0c-a1a1-c4b650235f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:27.456044 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f478223-b9a8-4f0c-a1a1-c4b650235f71 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:27.456383 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f478223-b9a8-4f0c-a1a1-c4b650235f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:27.456664 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f478223-b9a8-4f0c-a1a1-c4b650235f71 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:27.457321 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f478223-b9a8-4f0c-a1a1-c4b650235f71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:27.457626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f478223-b9a8-4f0c-a1a1-c4b650235f71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:27.459028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f478223-b9a8-4f0c-a1a1-c4b650235f71 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:27.473066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 305/615] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:27.479184 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75271387-0ff9-4894-a8e1-efac132748c6 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:27.481606 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f478223-b9a8-4f0c-a1a1-c4b650235f71 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:27.523861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:27.532720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:27.551415 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff478fe4-bb9c-4587-8929-7e0ee810ecac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:27.552059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff478fe4-bb9c-4587-8929-7e0ee810ecac None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:27.552595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff478fe4-bb9c-4587-8929-7e0ee810ecac None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:27.552595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff478fe4-bb9c-4587-8929-7e0ee810ecac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:27.595081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff478fe4-bb9c-4587-8929-7e0ee810ecac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-ff478fe4-bb9c-4587-8929-7e0ee810ecac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:27.601755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:27.602734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:27.603550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:27.603550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:27.603550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:27.603550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:27.603550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:27.603550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:27.603550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:27.603550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:27.603550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:27.603550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:27.603550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:27.603550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:27.603550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 311/616] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:27.613744 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f478223-b9a8-4f0c-a1a1-c4b650235f71 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:27.614337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff478fe4-bb9c-4587-8929-7e0ee810ecac None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:27.651425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e267a04-e6e5-4aba-9b8f-8003b66ad407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:27.652078 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e267a04-e6e5-4aba-9b8f-8003b66ad407 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:27.652292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e267a04-e6e5-4aba-9b8f-8003b66ad407 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:27.652561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e267a04-e6e5-4aba-9b8f-8003b66ad407 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:27.660502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e267a04-e6e5-4aba-9b8f-8003b66ad407 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:27.662052 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e267a04-e6e5-4aba-9b8f-8003b66ad407 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-2e267a04-e6e5-4aba-9b8f-8003b66ad407 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:27.676087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:27.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:27.681068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:27.681068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:27.681068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:27.681068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:27.681068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:27.681068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:27.681068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:27.681068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:27.681068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:53:27.681068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:27.681068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 306/617] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:53:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:27.686586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e267a04-e6e5-4aba-9b8f-8003b66ad407 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:27.813958 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:27.816175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff478fe4-bb9c-4587-8929-7e0ee810ecac None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:27.839455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e267a04-e6e5-4aba-9b8f-8003b66ad407 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:27.889187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee205b-cef6-4d99-9552-1169072f07e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:27.890955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee205b-cef6-4d99-9552-1169072f07e4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:27.891235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee205b-cef6-4d99-9552-1169072f07e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:27.891505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee205b-cef6-4d99-9552-1169072f07e4 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:27.892254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee205b-cef6-4d99-9552-1169072f07e4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:27.892557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee205b-cef6-4d99-9552-1169072f07e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:27.894384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee205b-cef6-4d99-9552-1169072f07e4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:27.902542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d82b9-0dcc-4935-89d1-fe8db1f51369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:27.903083 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d82b9-0dcc-4935-89d1-fe8db1f51369 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:27.903490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d82b9-0dcc-4935-89d1-fe8db1f51369 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:27.903757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d82b9-0dcc-4935-89d1-fe8db1f51369 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:27.904426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d82b9-0dcc-4935-89d1-fe8db1f51369 None admin] RBAC: Adding query filter params (name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:27.904726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d82b9-0dcc-4935-89d1-fe8db1f51369 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:27.908738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d82b9-0dcc-4935-89d1-fe8db1f51369 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:27.926677 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 312/618] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:27.930687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 307/619] 10.222.0.62 () {56 vars in 1135 bytes} [Fri Sep 9 11:53:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:27.935522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d82b9-0dcc-4935-89d1-fe8db1f51369 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:27.940165 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee205b-cef6-4d99-9552-1169072f07e4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:28.036959 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:28.108000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee205b-cef6-4d99-9552-1169072f07e4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:28.132423 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d82b9-0dcc-4935-89d1-fe8db1f51369 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:28.166847 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6b38d-8fb6-4c7f-a2f6-7d63341287c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:28.167401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6b38d-8fb6-4c7f-a2f6-7d63341287c5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:28.167673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6b38d-8fb6-4c7f-a2f6-7d63341287c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:28.167939 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6b38d-8fb6-4c7f-a2f6-7d63341287c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:28.190046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6b38d-8fb6-4c7f-a2f6-7d63341287c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:28.191512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6b38d-8fb6-4c7f-a2f6-7d63341287c5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:28.194013 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:28.204607 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d69623-96ca-4d03-ac94-14932d1af44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:28.205245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d69623-96ca-4d03-ac94-14932d1af44f None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:28.205523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d69623-96ca-4d03-ac94-14932d1af44f None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:28.205833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d69623-96ca-4d03-ac94-14932d1af44f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:28.217543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d69623-96ca-4d03-ac94-14932d1af44f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:28.219045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d69623-96ca-4d03-ac94-14932d1af44f None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-afe6b38d-8fb6-4c7f-a2f6-7d63341287c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:28.223552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:28.224507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:28.225929 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:28.225929 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:28.225929 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:28.232694 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 308/620] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:28.234169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6b38d-8fb6-4c7f-a2f6-7d63341287c5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-26d69623-96ca-4d03-ac94-14932d1af44f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:28.246529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:28.247522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:28.248456 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:28.248456 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:28.248456 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:28.248456 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 313/621] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:28.253439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d69623-96ca-4d03-ac94-14932d1af44f None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:28.420799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d69623-96ca-4d03-ac94-14932d1af44f None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:28.423830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6b38d-8fb6-4c7f-a2f6-7d63341287c5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:28.446486 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:28.488133 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6428691-91d8-41fc-b29e-78e545e554a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:28.488757 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6428691-91d8-41fc-b29e-78e545e554a0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:28.490012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6428691-91d8-41fc-b29e-78e545e554a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:28.490329 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6428691-91d8-41fc-b29e-78e545e554a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:28.490990 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6428691-91d8-41fc-b29e-78e545e554a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:28.491820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6428691-91d8-41fc-b29e-78e545e554a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:28.493193 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6428691-91d8-41fc-b29e-78e545e554a0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:28.496298 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4730c-d9cd-4558-bbb9-7fd56e8f7ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:28.496923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4730c-d9cd-4558-bbb9-7fd56e8f7ccb None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:28.497200 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4730c-d9cd-4558-bbb9-7fd56e8f7ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:28.497469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4730c-d9cd-4558-bbb9-7fd56e8f7ccb None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:28.498141 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4730c-d9cd-4558-bbb9-7fd56e8f7ccb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:28.498456 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4730c-d9cd-4558-bbb9-7fd56e8f7ccb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:28.499883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4730c-d9cd-4558-bbb9-7fd56e8f7ccb None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:28.521877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 309/622] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:28.530649 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6428691-91d8-41fc-b29e-78e545e554a0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:28.531103 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 314/623] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:28.542746 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4730c-d9cd-4558-bbb9-7fd56e8f7ccb None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:28.642959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:28.724974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6428691-91d8-41fc-b29e-78e545e554a0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:28.734593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4730c-d9cd-4558-bbb9-7fd56e8f7ccb None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:28.773593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fd625-d4af-49e9-a86c-42982be4752a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:28.774407 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fd625-d4af-49e9-a86c-42982be4752a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:28.774913 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fd625-d4af-49e9-a86c-42982be4752a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:28.776259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fd625-d4af-49e9-a86c-42982be4752a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:28.787394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fd625-d4af-49e9-a86c-42982be4752a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:28.788907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fd625-d4af-49e9-a86c-42982be4752a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-846fd625-d4af-49e9-a86c-42982be4752a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:28.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:28.804785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:28.807035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:28.807035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:28.807035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:28.807035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:28.807035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:28.807035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:28.807035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:28.807035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:28.807035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:53:28.807035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:28.807035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 310/624] 10.222.0.62 () {56 vars in 1093 bytes} [Fri Sep 9 11:53:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:28.815535 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fd625-d4af-49e9-a86c-42982be4752a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:28.838387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923f750-bab2-41eb-97b4-a3c117557db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:28.839119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923f750-bab2-41eb-97b4-a3c117557db7 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:28.839398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923f750-bab2-41eb-97b4-a3c117557db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:28.839668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923f750-bab2-41eb-97b4-a3c117557db7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:28.856236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923f750-bab2-41eb-97b4-a3c117557db7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:28.857711 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923f750-bab2-41eb-97b4-a3c117557db7 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:28.890705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-3923f750-bab2-41eb-97b4-a3c117557db7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:28.890705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:28.890705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:28.890705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:28.890705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:28.890705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:28.890705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:28.890705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:28.890705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:28.890705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:28.890705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:28.890705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:28.890705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:28.890705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:28.890705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:28.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:28.895127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:28.895127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:28.895127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:28.895127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:28.895127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:28.895127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:28.895127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:28.895127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:28.895127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 315/625] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:28.902140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923f750-bab2-41eb-97b4-a3c117557db7 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:28.955656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fd625-d4af-49e9-a86c-42982be4752a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:28.955895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:28.970590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:28.995602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b5980-743c-4050-ae80-9fd12009f6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:28.996293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b5980-743c-4050-ae80-9fd12009f6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:28.996570 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b5980-743c-4050-ae80-9fd12009f6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:28.997153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b5980-743c-4050-ae80-9fd12009f6f9 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:28.997882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b5980-743c-4050-ae80-9fd12009f6f9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:28.998223 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b5980-743c-4050-ae80-9fd12009f6f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:28.999631 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b5980-743c-4050-ae80-9fd12009f6f9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:29.018078 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 311/626] 10.222.0.62 () {56 vars in 1133 bytes} [Fri Sep 9 11:53:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:29.019305 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba673c-4d18-4938-a4b5-67d3bba39e57 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:29.019598 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba673c-4d18-4938-a4b5-67d3bba39e57 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:29.019870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba673c-4d18-4938-a4b5-67d3bba39e57 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:29.025239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 312/627] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:29.025239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610390d2-d081-465e-a2a2-0f4e54dd485a None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:29.025239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610390d2-d081-465e-a2a2-0f4e54dd485a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:29.025239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610390d2-d081-465e-a2a2-0f4e54dd485a None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:29.025239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 313/628] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:29.025651 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610390d2-d081-465e-a2a2-0f4e54dd485a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:29.047853 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923f750-bab2-41eb-97b4-a3c117557db7 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:29.087742 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09c399-eb66-4b12-b29c-0954957efb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:29.088369 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09c399-eb66-4b12-b29c-0954957efb3c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:29.088642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09c399-eb66-4b12-b29c-0954957efb3c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:29.088909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09c399-eb66-4b12-b29c-0954957efb3c None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:29.089658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf09c399-eb66-4b12-b29c-0954957efb3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:29.089940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf09c399-eb66-4b12-b29c-0954957efb3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:29.091445 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf09c399-eb66-4b12-b29c-0954957efb3c None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:29.106085 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 316/629] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:29.107585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaab326-1a2e-42e1-a062-a7e8fb80f2a1 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:29.107870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaab326-1a2e-42e1-a062-a7e8fb80f2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:29.108143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaab326-1a2e-42e1-a062-a7e8fb80f2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:29.129698 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-1eaab326-1a2e-42e1-a062-a7e8fb80f2a1 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:29.131005 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-1eaab326-1a2e-42e1-a062-a7e8fb80f2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:29.147892 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610390d2-d081-465e-a2a2-0f4e54dd485a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:29.191928 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b951d12-8e4e-4a2b-8324-0c13ae19e595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:29.192551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b951d12-8e4e-4a2b-8324-0c13ae19e595 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:29.192829 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b951d12-8e4e-4a2b-8324-0c13ae19e595 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:29.193100 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b951d12-8e4e-4a2b-8324-0c13ae19e595 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:29.231519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 317/630] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:29.232998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d8265-4ae1-47ac-b94d-ecd8b17a1c98 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:29.234186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d8265-4ae1-47ac-b94d-ecd8b17a1c98 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:29.234186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d8265-4ae1-47ac-b94d-ecd8b17a1c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:29.241042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:29.254479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-454d8265-4ae1-47ac-b94d-ecd8b17a1c98 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:29.255451 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-454d8265-4ae1-47ac-b94d-ecd8b17a1c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:29.267498 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:29.272200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b951d12-8e4e-4a2b-8324-0c13ae19e595 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:29.273938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b951d12-8e4e-4a2b-8324-0c13ae19e595 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:29.333277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 318/631] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:29.334223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d8265-4ae1-47ac-b94d-ecd8b17a1c98 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:29.384521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 314/632] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:53:29.386266 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6ec6e-a520-4a37-866c-0d8a9dece677 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:29.386781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6ec6e-a520-4a37-866c-0d8a9dece677 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:29.387257 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6ec6e-a520-4a37-866c-0d8a9dece677 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:29.388948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 315/633] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:29.394183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6ec6e-a520-4a37-866c-0d8a9dece677 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:29.459944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d8265-4ae1-47ac-b94d-ecd8b17a1c98 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:29.489436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:29.499108 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c2106-4cd0-4feb-8d6b-d2d965a2abe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:29.500185 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c2106-4cd0-4feb-8d6b-d2d965a2abe0 None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:29.500676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c2106-4cd0-4feb-8d6b-d2d965a2abe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:29.501237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c2106-4cd0-4feb-8d6b-d2d965a2abe0 None admin] PATH_INFO: `/v3/projects/32b06df0c5504592b012585f193386e7/users/1b2de52919e54bf9bcc352909c9f905f/roles/035f7d9f445d406eb2e83ca90fc4d5ca` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:29.517236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c2106-4cd0-4feb-8d6b-d2d965a2abe0 None admin] RBAC: Authorizing `identity:create_grant(project_id=32b06df0c5504592b012585f193386e7, user_id=1b2de52919e54bf9bcc352909c9f905f, role_id=035f7d9f445d406eb2e83ca90fc4d5ca)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:29.518829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c2106-4cd0-4feb-8d6b-d2d965a2abe0 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:29.522195 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:29.527619 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6ec6e-a520-4a37-866c-0d8a9dece677 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:29.533854 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 319/634] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:53:29 2022] PUT /identity/v3/projects/32b06df0c5504592b012585f193386e7/users/1b2de52919e54bf9bcc352909c9f905f/roles/035f7d9f445d406eb2e83ca90fc4d5ca => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:29.535184 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a3fbc-bc1b-4351-8476-cb2f11935ae4 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:29.535464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a3fbc-bc1b-4351-8476-cb2f11935ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:29.535731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a3fbc-bc1b-4351-8476-cb2f11935ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:29.563342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-5c8a3fbc-bc1b-4351-8476-cb2f11935ae4 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:29.565545 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-5c8a3fbc-bc1b-4351-8476-cb2f11935ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:29.572424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbc8f2-d613-43dc-9f4e-8ac5990d8df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:29.573731 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbc8f2-d613-43dc-9f4e-8ac5990d8df6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:29.574330 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbc8f2-d613-43dc-9f4e-8ac5990d8df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:29.574815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbc8f2-d613-43dc-9f4e-8ac5990d8df6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:29.589020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbc8f2-d613-43dc-9f4e-8ac5990d8df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:29.590909 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbc8f2-d613-43dc-9f4e-8ac5990d8df6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-c7bbc8f2-d613-43dc-9f4e-8ac5990d8df6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:29.611846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:29.613104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:29.614032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:29.614032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:29.614032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:29.614032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 316/635] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:29.614301 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbc8f2-d613-43dc-9f4e-8ac5990d8df6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:29.652115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 320/636] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:29.653012 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a3fbc-bc1b-4351-8476-cb2f11935ae4 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:29.738566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbc8f2-d613-43dc-9f4e-8ac5990d8df6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:29.772463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:29.785762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685e293-adaa-4f7e-a946-50234e1fdb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:29.786551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685e293-adaa-4f7e-a946-50234e1fdb4c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:29.786933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685e293-adaa-4f7e-a946-50234e1fdb4c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:29.787328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685e293-adaa-4f7e-a946-50234e1fdb4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:29.798972 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685e293-adaa-4f7e-a946-50234e1fdb4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:29.802663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a3fbc-bc1b-4351-8476-cb2f11935ae4 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-c685e293-adaa-4f7e-a946-50234e1fdb4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:29.805149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:29.806287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:29.807343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:29.807343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:29.807343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:29.807343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:29.807343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:29.807343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:29.807343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:29.807343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:29.807343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:29.807343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:29.807343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:29.807343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:29.807343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 317/637] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:29.808218 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685e293-adaa-4f7e-a946-50234e1fdb4c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:29.837730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:29.847587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43291814-745e-44f1-9a30-db8fdf95a54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:29.848338 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43291814-745e-44f1-9a30-db8fdf95a54b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:29.848683 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43291814-745e-44f1-9a30-db8fdf95a54b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:29.849055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43291814-745e-44f1-9a30-db8fdf95a54b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:29.860631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43291814-745e-44f1-9a30-db8fdf95a54b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-43291814-745e-44f1-9a30-db8fdf95a54b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:29.865673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:29.867663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:29.869519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:29.869519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:29.869519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:29.869519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:29.869519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:29.869519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:29.869519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:29.869519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:29.869519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:29.869519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:29.869519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:29.869519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:29.869519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 321/638] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:29.871192 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43291814-745e-44f1-9a30-db8fdf95a54b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:29.921774 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685e293-adaa-4f7e-a946-50234e1fdb4c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:29.963925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106bc87-c0bf-46dc-ba78-63e419f258fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:29.965012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106bc87-c0bf-46dc-ba78-63e419f258fc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:29.965384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106bc87-c0bf-46dc-ba78-63e419f258fc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:29.965729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106bc87-c0bf-46dc-ba78-63e419f258fc None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:29.966614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106bc87-c0bf-46dc-ba78-63e419f258fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:29.967301 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106bc87-c0bf-46dc-ba78-63e419f258fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:29.968677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106bc87-c0bf-46dc-ba78-63e419f258fc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:29.980948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 318/639] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:29.982147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106bc87-c0bf-46dc-ba78-63e419f258fc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:29.992155 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43291814-745e-44f1-9a30-db8fdf95a54b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:30.027418 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e49a20-48d6-4447-88dc-45a8fad4a410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:30.027949 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e49a20-48d6-4447-88dc-45a8fad4a410 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:30.028650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e49a20-48d6-4447-88dc-45a8fad4a410 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:30.028932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e49a20-48d6-4447-88dc-45a8fad4a410 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:30.029615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e49a20-48d6-4447-88dc-45a8fad4a410 None admin] RBAC: Adding query filter params (name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:30.029918 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e49a20-48d6-4447-88dc-45a8fad4a410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:30.032012 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e49a20-48d6-4447-88dc-45a8fad4a410 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:30.034567 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:30.042171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 322/640] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:30.043520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e49a20-48d6-4447-88dc-45a8fad4a410 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:30.099184 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106bc87-c0bf-46dc-ba78-63e419f258fc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:30.103232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:30.145377 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8ebdb-46f8-44b4-960e-533a6e2ce338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:30.145987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8ebdb-46f8-44b4-960e-533a6e2ce338 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:30.146289 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8ebdb-46f8-44b4-960e-533a6e2ce338 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:30.146558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8ebdb-46f8-44b4-960e-533a6e2ce338 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:30.147313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8ebdb-46f8-44b4-960e-533a6e2ce338 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:30.148162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8ebdb-46f8-44b4-960e-533a6e2ce338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:30.149516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8ebdb-46f8-44b4-960e-533a6e2ce338 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:30.162718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 319/641] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:30.163682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8ebdb-46f8-44b4-960e-533a6e2ce338 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:30.167178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e49a20-48d6-4447-88dc-45a8fad4a410 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:30.202543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30e3c6-7760-4319-a2c8-8e2612daf11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:30.203175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30e3c6-7760-4319-a2c8-8e2612daf11b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:30.203453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30e3c6-7760-4319-a2c8-8e2612daf11b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:30.203722 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30e3c6-7760-4319-a2c8-8e2612daf11b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:30.216699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30e3c6-7760-4319-a2c8-8e2612daf11b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-4c30e3c6-7760-4319-a2c8-8e2612daf11b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:30.222149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:30.223145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:30.227230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:30.227230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:30.227230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:30.227230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:30.227230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:30.227230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:30.227230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:30.227230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:30.227230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:30.227230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:30.227230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:30.227230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:30.227230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 323/642] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:30.227230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30e3c6-7760-4319-a2c8-8e2612daf11b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:30.285966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8ebdb-46f8-44b4-960e-533a6e2ce338 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:30.293968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:30.327122 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff08cac-548a-43a4-830c-5a6f1f3b22ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:30.327767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff08cac-548a-43a4-830c-5a6f1f3b22ae None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:30.328040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff08cac-548a-43a4-830c-5a6f1f3b22ae None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:30.328314 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff08cac-548a-43a4-830c-5a6f1f3b22ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:30.333356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30e3c6-7760-4319-a2c8-8e2612daf11b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:30.336803 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff08cac-548a-43a4-830c-5a6f1f3b22ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:30.337750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff08cac-548a-43a4-830c-5a6f1f3b22ae None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:30.343696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:30.352189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-7ff08cac-548a-43a4-830c-5a6f1f3b22ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:30.352189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:30.352189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:30.352189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:30.352189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:30.352189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:30.352189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:30.352189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:30.352189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:30.352189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:30.352189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:30.352189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:30.352189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:30.352189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:30.352189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:30.353210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:30.354169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:30.354169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:30.354169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:30.354169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:30.354169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:30.354169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:30.354169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:30.354169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:30.354169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 320/643] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:30.355343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff08cac-548a-43a4-830c-5a6f1f3b22ae None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:30.375495 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08767175-75fc-4d20-a65d-114f5004c7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:30.376185 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08767175-75fc-4d20-a65d-114f5004c7dd None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:30.376632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08767175-75fc-4d20-a65d-114f5004c7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:30.376909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08767175-75fc-4d20-a65d-114f5004c7dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:30.388669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08767175-75fc-4d20-a65d-114f5004c7dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:30.390369 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08767175-75fc-4d20-a65d-114f5004c7dd None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-08767175-75fc-4d20-a65d-114f5004c7dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:30.406029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:30.407130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:30.408630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:30.408630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:30.408630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:30.408630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 324/644] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:30.408630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08767175-75fc-4d20-a65d-114f5004c7dd None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:30.464752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff08cac-548a-43a4-830c-5a6f1f3b22ae None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:30.505634 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2d309-525b-4683-8045-bddbcf615ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:30.506386 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2d309-525b-4683-8045-bddbcf615ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:30.506674 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2d309-525b-4683-8045-bddbcf615ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:30.507546 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2d309-525b-4683-8045-bddbcf615ee7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:30.518058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2d309-525b-4683-8045-bddbcf615ee7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:30.519539 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2d309-525b-4683-8045-bddbcf615ee7 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:30.520972 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08767175-75fc-4d20-a65d-114f5004c7dd None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-88f2d309-525b-4683-8045-bddbcf615ee7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:30.535502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:30.536687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:30.537597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:30.537597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:30.537597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:30.537597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 321/645] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:30.537597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2d309-525b-4683-8045-bddbcf615ee7 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:30.560089 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:30.566285 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4195f-7e03-4ec1-bb8e-73574d0fd159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:30.567090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4195f-7e03-4ec1-bb8e-73574d0fd159 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:30.567368 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4195f-7e03-4ec1-bb8e-73574d0fd159 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:30.567629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4195f-7e03-4ec1-bb8e-73574d0fd159 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:30.568292 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4195f-7e03-4ec1-bb8e-73574d0fd159 None admin] RBAC: Adding query filter params (name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:30.568592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4195f-7e03-4ec1-bb8e-73574d0fd159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:30.569982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4195f-7e03-4ec1-bb8e-73574d0fd159 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:30.582748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 325/646] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:30.582748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4195f-7e03-4ec1-bb8e-73574d0fd159 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:30.607816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:30.651411 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2d309-525b-4683-8045-bddbcf615ee7 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:30.687043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ff15d-0bfc-4674-81ae-957e253d8dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:30.687648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ff15d-0bfc-4674-81ae-957e253d8dbc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:30.687870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4195f-7e03-4ec1-bb8e-73574d0fd159 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:30.688002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ff15d-0bfc-4674-81ae-957e253d8dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:30.688164 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ff15d-0bfc-4674-81ae-957e253d8dbc None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:30.688846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ff15d-0bfc-4674-81ae-957e253d8dbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:30.689145 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ff15d-0bfc-4674-81ae-957e253d8dbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:30.690587 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ff15d-0bfc-4674-81ae-957e253d8dbc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:30.707214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 322/647] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:30.707214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ff15d-0bfc-4674-81ae-957e253d8dbc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:30.731426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758b2ab-45d6-4c52-adf5-1ad2b5322ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:30.732041 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758b2ab-45d6-4c52-adf5-1ad2b5322ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:30.732332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758b2ab-45d6-4c52-adf5-1ad2b5322ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:30.732589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758b2ab-45d6-4c52-adf5-1ad2b5322ac8 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:30.733236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758b2ab-45d6-4c52-adf5-1ad2b5322ac8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:30.733546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758b2ab-45d6-4c52-adf5-1ad2b5322ac8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:30.735202 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758b2ab-45d6-4c52-adf5-1ad2b5322ac8 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:30.751796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 326/648] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:30.751796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758b2ab-45d6-4c52-adf5-1ad2b5322ac8 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:30.791924 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:30.815678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ff15d-0bfc-4674-81ae-957e253d8dbc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:30.839390 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:30.853264 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707c3d5-aca3-42cd-9d46-b7702bcc6c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:30.853763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707c3d5-aca3-42cd-9d46-b7702bcc6c45 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:30.854152 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707c3d5-aca3-42cd-9d46-b7702bcc6c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:30.854521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707c3d5-aca3-42cd-9d46-b7702bcc6c45 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:30.855258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707c3d5-aca3-42cd-9d46-b7702bcc6c45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:30.855670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707c3d5-aca3-42cd-9d46-b7702bcc6c45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:30.857123 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707c3d5-aca3-42cd-9d46-b7702bcc6c45 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:30.859785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758b2ab-45d6-4c52-adf5-1ad2b5322ac8 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:30.874006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 323/649] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:30.874006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707c3d5-aca3-42cd-9d46-b7702bcc6c45 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:30.902027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c45e2a-28bd-4c84-a21a-38b778cc2a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:30.902692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c45e2a-28bd-4c84-a21a-38b778cc2a8f None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:30.902969 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c45e2a-28bd-4c84-a21a-38b778cc2a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:30.903272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c45e2a-28bd-4c84-a21a-38b778cc2a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:30.916199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c45e2a-28bd-4c84-a21a-38b778cc2a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:30.917756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c45e2a-28bd-4c84-a21a-38b778cc2a8f None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-c8c45e2a-28bd-4c84-a21a-38b778cc2a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:30.933635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:30.934731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:30.935986 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:30.935986 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:30.935986 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:30.935986 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 327/650] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:30.936714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c45e2a-28bd-4c84-a21a-38b778cc2a8f None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:30.991688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707c3d5-aca3-42cd-9d46-b7702bcc6c45 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:31.027963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42529037-2066-4cb8-b5e6-620d94870f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:31.028895 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42529037-2066-4cb8-b5e6-620d94870f59 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:31.029382 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42529037-2066-4cb8-b5e6-620d94870f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:31.029850 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42529037-2066-4cb8-b5e6-620d94870f59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:31.030825 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42529037-2066-4cb8-b5e6-620d94870f59 None admin] RBAC: Adding query filter params (role.id=035f7d9f445d406eb2e83ca90fc4d5ca, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=1b30e80c421245b5a1ed5849b28ea90e) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:31.031346 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42529037-2066-4cb8-b5e6-620d94870f59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:31.032981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42529037-2066-4cb8-b5e6-620d94870f59 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:31.039992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 324/651] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:30 2022] GET /identity/v3/role_assignments?user.id=1b30e80c421245b5a1ed5849b28ea90e&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=035f7d9f445d406eb2e83ca90fc4d5ca => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:31.041217 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42529037-2066-4cb8-b5e6-620d94870f59 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:31.051540 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:31.062026 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c45e2a-28bd-4c84-a21a-38b778cc2a8f None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:31.101250 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b60d92-73ae-405d-9230-0755ac6a9649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:31.102257 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b60d92-73ae-405d-9230-0755ac6a9649 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:31.102636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b60d92-73ae-405d-9230-0755ac6a9649 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:31.102997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b60d92-73ae-405d-9230-0755ac6a9649 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:31.105408 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:31.113730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b60d92-73ae-405d-9230-0755ac6a9649 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:31.115543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b60d92-73ae-405d-9230-0755ac6a9649 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-a6b60d92-73ae-405d-9230-0755ac6a9649 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:31.129777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:31.131090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:31.132349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:31.132349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:31.132349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:31.132349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:31.132349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:31.132349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:31.132349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:31.132349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:31.132349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 11:53:31.132349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:31.138568 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 328/652] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:31.139746 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b60d92-73ae-405d-9230-0755ac6a9649 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:31.182076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42529037-2066-4cb8-b5e6-620d94870f59 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:31.238412 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01263c4f-e6f5-461f-bf83-f60aa6ecb57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:31.239232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01263c4f-e6f5-461f-bf83-f60aa6ecb57e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:31.239622 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01263c4f-e6f5-461f-bf83-f60aa6ecb57e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:31.239990 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01263c4f-e6f5-461f-bf83-f60aa6ecb57e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:31.251268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01263c4f-e6f5-461f-bf83-f60aa6ecb57e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:31.252989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01263c4f-e6f5-461f-bf83-f60aa6ecb57e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-01263c4f-e6f5-461f-bf83-f60aa6ecb57e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:31.268547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:31.269552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:31.270693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:31.270693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:31.270693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:31.270693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:31.270693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:31.270693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:31.270693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:31.270693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:31.270693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:53:31.270693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:31.270693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 325/653] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:31.271280 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01263c4f-e6f5-461f-bf83-f60aa6ecb57e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:31.275082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b60d92-73ae-405d-9230-0755ac6a9649 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:31.325217 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a562338-3b9e-4388-8567-1e1b3ffa6c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:31.325758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a562338-3b9e-4388-8567-1e1b3ffa6c3b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:31.326030 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a562338-3b9e-4388-8567-1e1b3ffa6c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:31.326317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a562338-3b9e-4388-8567-1e1b3ffa6c3b None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:31.326968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a562338-3b9e-4388-8567-1e1b3ffa6c3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:31.327264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a562338-3b9e-4388-8567-1e1b3ffa6c3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:31.328637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a562338-3b9e-4388-8567-1e1b3ffa6c3b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:31.349225 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 329/654] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:31.354729 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a562338-3b9e-4388-8567-1e1b3ffa6c3b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:31.369144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:31.412752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01263c4f-e6f5-461f-bf83-f60aa6ecb57e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:31.433212 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:31.458038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3880b-0be0-4a87-b6f3-d2998768f8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:31.458674 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3880b-0be0-4a87-b6f3-d2998768f8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:31.458948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3880b-0be0-4a87-b6f3-d2998768f8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:31.459214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3880b-0be0-4a87-b6f3-d2998768f8d4 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:31.460511 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3880b-0be0-4a87-b6f3-d2998768f8d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:31.460809 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3880b-0be0-4a87-b6f3-d2998768f8d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:31.462298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3880b-0be0-4a87-b6f3-d2998768f8d4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:31.484900 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 326/655] 10.222.0.62 () {56 vars in 1137 bytes} [Fri Sep 9 11:53:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:31.489893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3880b-0be0-4a87-b6f3-d2998768f8d4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:31.504570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a562338-3b9e-4388-8567-1e1b3ffa6c3b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:31.544830 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a9a7a-1312-4732-82c4-193ecf6ebfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:31.545421 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a9a7a-1312-4732-82c4-193ecf6ebfe0 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:31.545694 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a9a7a-1312-4732-82c4-193ecf6ebfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:31.546011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a9a7a-1312-4732-82c4-193ecf6ebfe0 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:31.546799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a9a7a-1312-4732-82c4-193ecf6ebfe0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:31.547099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a9a7a-1312-4732-82c4-193ecf6ebfe0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:31.548611 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a9a7a-1312-4732-82c4-193ecf6ebfe0 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:31.574107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 330/656] 10.222.0.62 () {56 vars in 1137 bytes} [Fri Sep 9 11:53:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:31.574107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a9a7a-1312-4732-82c4-193ecf6ebfe0 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:31.636318 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3880b-0be0-4a87-b6f3-d2998768f8d4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:31.661292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:31.679678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dab830-6d29-4ea9-8e03-446a65a1c962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:31.680687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dab830-6d29-4ea9-8e03-446a65a1c962 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:31.680869 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a9a7a-1312-4732-82c4-193ecf6ebfe0 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:31.681336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dab830-6d29-4ea9-8e03-446a65a1c962 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:31.681828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dab830-6d29-4ea9-8e03-446a65a1c962 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:31.689867 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dab830-6d29-4ea9-8e03-446a65a1c962 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:31.691639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dab830-6d29-4ea9-8e03-446a65a1c962 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:31.695980 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-75dab830-6d29-4ea9-8e03-446a65a1c962 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:31.708183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:31.709172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:31.710718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:31.710718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:31.710718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:31.710718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:31.710718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:31.710718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:31.710718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:31.710718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:31.710718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 11:53:31.710718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:31.716844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 327/657] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:31.716844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dab830-6d29-4ea9-8e03-446a65a1c962 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:31.731827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0868325-7dd7-4f2b-942e-60cefc135e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:31.732885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0868325-7dd7-4f2b-942e-60cefc135e26 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:31.733367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0868325-7dd7-4f2b-942e-60cefc135e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:31.734360 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0868325-7dd7-4f2b-942e-60cefc135e26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:31.817125 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0868325-7dd7-4f2b-942e-60cefc135e26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:31.818797 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0868325-7dd7-4f2b-942e-60cefc135e26 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:31.829516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dab830-6d29-4ea9-8e03-446a65a1c962 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:31.864115 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ba814-cb9f-456c-a953-ccc5405a593c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:31.864730 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ba814-cb9f-456c-a953-ccc5405a593c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:31.865065 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ba814-cb9f-456c-a953-ccc5405a593c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:31.865334 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ba814-cb9f-456c-a953-ccc5405a593c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:31.927021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:31.942871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 331/658] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:53:31.943721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0868325-7dd7-4f2b-942e-60cefc135e26 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:31.958725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ba814-cb9f-456c-a953-ccc5405a593c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:31.960704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:31.960824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ba814-cb9f-456c-a953-ccc5405a593c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:32.055204 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0868325-7dd7-4f2b-942e-60cefc135e26 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:32.068051 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 328/659] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:53:32.069427 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61014262-c65d-402e-b2ec-473196edafb1 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:32.069714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61014262-c65d-402e-b2ec-473196edafb1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:32.069983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61014262-c65d-402e-b2ec-473196edafb1 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:32.075513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 329/660] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:32.075513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61014262-c65d-402e-b2ec-473196edafb1 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:32.103827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7eb55-f7b3-437a-b540-3071c1e49b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:32.104538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7eb55-f7b3-437a-b540-3071c1e49b5d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:32.104880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7eb55-f7b3-437a-b540-3071c1e49b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:32.105221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7eb55-f7b3-437a-b540-3071c1e49b5d None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:32.106033 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7eb55-f7b3-437a-b540-3071c1e49b5d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:32.106438 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7eb55-f7b3-437a-b540-3071c1e49b5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:32.107967 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7eb55-f7b3-437a-b540-3071c1e49b5d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:32.130873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 332/661] 10.222.0.62 () {56 vars in 1139 bytes} [Fri Sep 9 11:53:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:32.131769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7eb55-f7b3-437a-b540-3071c1e49b5d None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:32.165050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:32.188411 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61014262-c65d-402e-b2ec-473196edafb1 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:32.204701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:32.231599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabbe82-3223-4bfe-bb42-9b8777869260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:32.232290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabbe82-3223-4bfe-bb42-9b8777869260 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:32.232574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabbe82-3223-4bfe-bb42-9b8777869260 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:32.232843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabbe82-3223-4bfe-bb42-9b8777869260 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:32.243587 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabbe82-3223-4bfe-bb42-9b8777869260 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:32.244860 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7eb55-f7b3-437a-b540-3071c1e49b5d None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:32.245027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabbe82-3223-4bfe-bb42-9b8777869260 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-2dabbe82-3223-4bfe-bb42-9b8777869260 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:32.260016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:32.261012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:32.261910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:32.261910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:32.261910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:32.261910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 330/662] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:32.262455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee8f36-3f3e-429f-9e7a-df88fdea8462 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:32.262550 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee8f36-3f3e-429f-9e7a-df88fdea8462 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:32.262821 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee8f36-3f3e-429f-9e7a-df88fdea8462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:32.283322 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f5ee8f36-3f3e-429f-9e7a-df88fdea8462 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:32.285010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f5ee8f36-3f3e-429f-9e7a-df88fdea8462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:32.289886 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e241c7-1fb0-451e-8ac7-fc00fdf1249e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:32.290534 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e241c7-1fb0-451e-8ac7-fc00fdf1249e None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:32.290811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e241c7-1fb0-451e-8ac7-fc00fdf1249e None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:32.291075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e241c7-1fb0-451e-8ac7-fc00fdf1249e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:32.301728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e241c7-1fb0-451e-8ac7-fc00fdf1249e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:32.303214 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e241c7-1fb0-451e-8ac7-fc00fdf1249e None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-b0e241c7-1fb0-451e-8ac7-fc00fdf1249e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:32.318303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:32.319858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:32.320784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:32.320784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:32.320784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:32.320784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 333/663] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:32.320784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e241c7-1fb0-451e-8ac7-fc00fdf1249e None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:32.363238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 331/664] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:32.364384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee8f36-3f3e-429f-9e7a-df88fdea8462 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:32.428362 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:32.451608 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e241c7-1fb0-451e-8ac7-fc00fdf1249e None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:32.471519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee8f36-3f3e-429f-9e7a-df88fdea8462 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:32.482639 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:32.501076 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34cee2-6faf-46cf-8345-99580cfaad1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:32.501954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34cee2-6faf-46cf-8345-99580cfaad1e None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:32.502265 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34cee2-6faf-46cf-8345-99580cfaad1e None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:32.502545 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34cee2-6faf-46cf-8345-99580cfaad1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:32.516559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a6906-8cd5-4af8-bcc4-25a7ab1bab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:32.517505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a6906-8cd5-4af8-bcc4-25a7ab1bab01 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:32.517982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a6906-8cd5-4af8-bcc4-25a7ab1bab01 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:32.518561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a6906-8cd5-4af8-bcc4-25a7ab1bab01 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:32.519401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a6906-8cd5-4af8-bcc4-25a7ab1bab01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:32.519899 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a6906-8cd5-4af8-bcc4-25a7ab1bab01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:32.521495 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a6906-8cd5-4af8-bcc4-25a7ab1bab01 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:32.535125 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 332/665] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:32.536385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a6906-8cd5-4af8-bcc4-25a7ab1bab01 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:32.594608 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34cee2-6faf-46cf-8345-99580cfaad1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:32.597191 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34cee2-6faf-46cf-8345-99580cfaad1e None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:32.663865 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a6906-8cd5-4af8-bcc4-25a7ab1bab01 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:32.692233 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:32.706480 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 334/666] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:53:32.706625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633a18d-20ab-44ac-82f8-01cb579043bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:32.707049 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633a18d-20ab-44ac-82f8-01cb579043bc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:32.707325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633a18d-20ab-44ac-82f8-01cb579043bc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:32.707670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633a18d-20ab-44ac-82f8-01cb579043bc None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:32.708323 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633a18d-20ab-44ac-82f8-01cb579043bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:32.708613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34cee2-6faf-46cf-8345-99580cfaad1e None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:32.708718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633a18d-20ab-44ac-82f8-01cb579043bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:32.710028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633a18d-20ab-44ac-82f8-01cb579043bc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:32.722729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 333/667] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:32.723655 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633a18d-20ab-44ac-82f8-01cb579043bc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:32.735905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:32.820508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34cee2-6faf-46cf-8345-99580cfaad1e None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:32.833970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633a18d-20ab-44ac-82f8-01cb579043bc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:32.854926 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fba4b2-3c65-4ed3-afcf-66e41f980967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:32.855553 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fba4b2-3c65-4ed3-afcf-66e41f980967 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:32.855829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fba4b2-3c65-4ed3-afcf-66e41f980967 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:32.856098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fba4b2-3c65-4ed3-afcf-66e41f980967 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:32.863283 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fba4b2-3c65-4ed3-afcf-66e41f980967 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:32.864717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fba4b2-3c65-4ed3-afcf-66e41f980967 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:32.874510 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecca8b7-1f44-427d-a1e5-c7700fd1492e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:32.875394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecca8b7-1f44-427d-a1e5-c7700fd1492e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:32.875843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecca8b7-1f44-427d-a1e5-c7700fd1492e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:32.876288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecca8b7-1f44-427d-a1e5-c7700fd1492e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:32.876573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-f6fba4b2-3c65-4ed3-afcf-66e41f980967 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:32.876573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:32.876573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:32.876573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:32.876573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:32.876573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:32.876573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:32.876573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:32.876573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:32.876573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:32.876573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:32.876573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:32.876573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:32.876573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:32.876573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:32.877430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:32.878351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:32.878351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:32.878351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:32.878351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:32.878351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:32.878351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:32.878351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:32.878351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:32.878351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 335/668] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:32.878820 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fba4b2-3c65-4ed3-afcf-66e41f980967 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:32.891980 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecca8b7-1f44-427d-a1e5-c7700fd1492e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-fecca8b7-1f44-427d-a1e5-c7700fd1492e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:32.896877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:32.898349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:32.899188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:32.899188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:32.899188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:32.899188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:32.899188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:32.899188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:32.899188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:32.899188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:32.899188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:32.899188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:32.899188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:32.899188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:32.899188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 334/669] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:32.899867 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecca8b7-1f44-427d-a1e5-c7700fd1492e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:32.946748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:32.983871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:33.004173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fba4b2-3c65-4ed3-afcf-66e41f980967 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:33.016437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecca8b7-1f44-427d-a1e5-c7700fd1492e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:33.041214 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c17acc-6c5e-406c-8c59-f718874971b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:33.041855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c17acc-6c5e-406c-8c59-f718874971b3 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:33.042142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c17acc-6c5e-406c-8c59-f718874971b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:33.042421 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c17acc-6c5e-406c-8c59-f718874971b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:33.052497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c17acc-6c5e-406c-8c59-f718874971b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:33.053647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea48e57-463a-49c9-9249-29219d9301d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:33.053986 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c17acc-6c5e-406c-8c59-f718874971b3 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:33.054316 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea48e57-463a-49c9-9249-29219d9301d8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:33.054599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea48e57-463a-49c9-9249-29219d9301d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:33.054867 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea48e57-463a-49c9-9249-29219d9301d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:33.062094 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea48e57-463a-49c9-9249-29219d9301d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:33.063545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea48e57-463a-49c9-9249-29219d9301d8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-24c17acc-6c5e-406c-8c59-f718874971b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:33.070156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:33.071255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:33.072353 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:33.072353 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:33.072353 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:33.077214 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 336/670] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:33.077214 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c17acc-6c5e-406c-8c59-f718874971b3 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:33.077432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-fea48e57-463a-49c9-9249-29219d9301d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:33.077432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:33.077432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:33.077432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:33.077432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:33.077432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:33.077432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:33.077432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:33.077432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:33.077432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:33.077432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:33.077432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:33.077432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:33.077432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:33.077432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:33.078434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:33.080597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:33.080597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:33.080597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:33.080597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:33.080597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:33.080597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:33.080597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:33.080597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:33.080597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 335/671] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:33.080597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea48e57-463a-49c9-9249-29219d9301d8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:33.201100 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea48e57-463a-49c9-9249-29219d9301d8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:33.201299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c17acc-6c5e-406c-8c59-f718874971b3 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:33.202981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:33.243084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881255f0-5880-4302-aa1b-a43ec7b9c037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:33.243084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881255f0-5880-4302-aa1b-a43ec7b9c037 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:33.243399 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881255f0-5880-4302-aa1b-a43ec7b9c037 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:33.243496 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881255f0-5880-4302-aa1b-a43ec7b9c037 None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:33.244168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881255f0-5880-4302-aa1b-a43ec7b9c037 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:33.244505 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881255f0-5880-4302-aa1b-a43ec7b9c037 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:33.245887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881255f0-5880-4302-aa1b-a43ec7b9c037 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:33.247829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:33.252099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0533336-dbdd-405c-9233-373d4b475b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:33.252719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0533336-dbdd-405c-9233-373d4b475b09 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:33.252995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0533336-dbdd-405c-9233-373d4b475b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:33.253262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0533336-dbdd-405c-9233-373d4b475b09 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:33.253923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0533336-dbdd-405c-9233-373d4b475b09 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:33.254259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0533336-dbdd-405c-9233-373d4b475b09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:33.255622 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0533336-dbdd-405c-9233-373d4b475b09 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:33.262543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 337/672] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:33.263990 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881255f0-5880-4302-aa1b-a43ec7b9c037 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:33.265638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 336/673] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:33.266515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0533336-dbdd-405c-9233-373d4b475b09 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:33.373380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0533336-dbdd-405c-9233-373d4b475b09 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:33.388543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881255f0-5880-4302-aa1b-a43ec7b9c037 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:33.410416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ec899-3767-4847-bb22-0aa9eb072165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:33.411018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ec899-3767-4847-bb22-0aa9eb072165 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:33.411292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ec899-3767-4847-bb22-0aa9eb072165 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:33.411562 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ec899-3767-4847-bb22-0aa9eb072165 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:33.412243 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ec899-3767-4847-bb22-0aa9eb072165 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:33.412568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ec899-3767-4847-bb22-0aa9eb072165 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:33.414135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ec899-3767-4847-bb22-0aa9eb072165 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:33.424281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9180800-8003-4b78-9cea-02967c8fa582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:33.424888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9180800-8003-4b78-9cea-02967c8fa582 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:33.425166 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9180800-8003-4b78-9cea-02967c8fa582 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:33.425438 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9180800-8003-4b78-9cea-02967c8fa582 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:33.431868 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9180800-8003-4b78-9cea-02967c8fa582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:33.432043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 337/674] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:33.432161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9180800-8003-4b78-9cea-02967c8fa582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:33.432240 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ec899-3767-4847-bb22-0aa9eb072165 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:33.432312 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9180800-8003-4b78-9cea-02967c8fa582 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:33.440414 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 338/675] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:33.441257 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9180800-8003-4b78-9cea-02967c8fa582 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:33.453952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:33.496446 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:33.531459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ec899-3767-4847-bb22-0aa9eb072165 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:33.541905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9180800-8003-4b78-9cea-02967c8fa582 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:33.566064 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db443882-fa89-4945-a8b5-6663383bcfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:33.566692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db443882-fa89-4945-a8b5-6663383bcfd6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:33.566966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db443882-fa89-4945-a8b5-6663383bcfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:33.567235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db443882-fa89-4945-a8b5-6663383bcfd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:33.567947 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db443882-fa89-4945-a8b5-6663383bcfd6 None admin] RBAC: Adding query filter params (role.id=035f7d9f445d406eb2e83ca90fc4d5ca, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=2d477a49a1724b7e8a4699a5ebd04bde) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:33.568301 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db443882-fa89-4945-a8b5-6663383bcfd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:33.570238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db443882-fa89-4945-a8b5-6663383bcfd6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:33.575816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 338/676] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:33 2022] GET /identity/v3/role_assignments?user.id=2d477a49a1724b7e8a4699a5ebd04bde&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=035f7d9f445d406eb2e83ca90fc4d5ca => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:33.576692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc27ae-f21a-4cdd-9313-e703312ce4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:33.576950 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a3ae3-39d8-476a-9c41-d30774d1810d None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:33.577227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a3ae3-39d8-476a-9c41-d30774d1810d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:33.577327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc27ae-f21a-4cdd-9313-e703312ce4ef None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:33.577516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a3ae3-39d8-476a-9c41-d30774d1810d None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:33.577614 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc27ae-f21a-4cdd-9313-e703312ce4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:33.577846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc27ae-f21a-4cdd-9313-e703312ce4ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:33.578857 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 339/677] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:33.579380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a3ae3-39d8-476a-9c41-d30774d1810d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:33.593316 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc27ae-f21a-4cdd-9313-e703312ce4ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:33.594856 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc27ae-f21a-4cdd-9313-e703312ce4ef None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-8bbc27ae-f21a-4cdd-9313-e703312ce4ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:33.611325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:33.612361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:33.618924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:33.618924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:33.618924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:33.618924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 339/678] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:33.618924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc27ae-f21a-4cdd-9313-e703312ce4ef None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:33.702511 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:33.718904 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a3ae3-39d8-476a-9c41-d30774d1810d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:33.724509 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc27ae-f21a-4cdd-9313-e703312ce4ef None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:33.732271 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:33.766256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e6e39-6e22-4eab-abcc-f3287a7a6f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:33.766908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e6e39-6e22-4eab-abcc-f3287a7a6f24 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:33.767188 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e6e39-6e22-4eab-abcc-f3287a7a6f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:33.767456 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e6e39-6e22-4eab-abcc-f3287a7a6f24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:33.768340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8361d54-c366-4367-9685-7b2b4a5119ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:33.768955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8361d54-c366-4367-9685-7b2b4a5119ed None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:33.769229 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8361d54-c366-4367-9685-7b2b4a5119ed None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:33.769576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8361d54-c366-4367-9685-7b2b4a5119ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:33.770252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8361d54-c366-4367-9685-7b2b4a5119ed None admin] RBAC: Adding query filter params (role.id=035f7d9f445d406eb2e83ca90fc4d5ca, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=a25411d51a524f4b9f0ff846a0b29b6f) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:33.770765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8361d54-c366-4367-9685-7b2b4a5119ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:33.772005 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8361d54-c366-4367-9685-7b2b4a5119ed None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:33.775205 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e6e39-6e22-4eab-abcc-f3287a7a6f24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:33.777551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e6e39-6e22-4eab-abcc-f3287a7a6f24 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:33.778271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 340/679] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:33 2022] GET /identity/v3/role_assignments?user.id=a25411d51a524f4b9f0ff846a0b29b6f&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=035f7d9f445d406eb2e83ca90fc4d5ca => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:33.779536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446798a-2e99-407e-a0ce-1fc610c787af None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:33.779821 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446798a-2e99-407e-a0ce-1fc610c787af None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:33.780091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446798a-2e99-407e-a0ce-1fc610c787af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:33.791900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-917e6e39-6e22-4eab-abcc-f3287a7a6f24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:33.791900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:33.791900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:33.791900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:33.791900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:33.791900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:33.791900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:33.791900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:33.791900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:33.791900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:33.791900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:33.791900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:33.791900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:33.791900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:33.791900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:33.792923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:33.793808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:33.793808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:33.793808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:33.793808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:33.793808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:33.793808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:33.793808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:33.793808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:33.796409 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 340/680] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:33.797577 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e6e39-6e22-4eab-abcc-f3287a7a6f24 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:33.801373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a446798a-2e99-407e-a0ce-1fc610c787af None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:33.803085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a446798a-2e99-407e-a0ce-1fc610c787af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:33.904856 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 341/681] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:33.905808 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446798a-2e99-407e-a0ce-1fc610c787af None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:33.936035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e6e39-6e22-4eab-abcc-f3287a7a6f24 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:33.973533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bede7c1-0db8-4f24-94c7-41677129fa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:33.974662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bede7c1-0db8-4f24-94c7-41677129fa04 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:33.975152 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bede7c1-0db8-4f24-94c7-41677129fa04 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:33.976165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bede7c1-0db8-4f24-94c7-41677129fa04 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:33.977082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bede7c1-0db8-4f24-94c7-41677129fa04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:33.977628 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bede7c1-0db8-4f24-94c7-41677129fa04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:33.979323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bede7c1-0db8-4f24-94c7-41677129fa04 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:33.993788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 341/682] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:34.000330 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bede7c1-0db8-4f24-94c7-41677129fa04 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:34.015185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:34.037522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:34.042747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446798a-2e99-407e-a0ce-1fc610c787af None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:34.083255 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2d543-0983-4195-b854-8edfc6da67bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:34.084353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2d543-0983-4195-b854-8edfc6da67bc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:34.085428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2d543-0983-4195-b854-8edfc6da67bc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:34.085906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2d543-0983-4195-b854-8edfc6da67bc None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:34.086860 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2d543-0983-4195-b854-8edfc6da67bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:34.087367 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2d543-0983-4195-b854-8edfc6da67bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:34.089786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2d543-0983-4195-b854-8edfc6da67bc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:34.103351 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 342/683] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:34.109182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2d543-0983-4195-b854-8edfc6da67bc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:34.144226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bede7c1-0db8-4f24-94c7-41677129fa04 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:34.193946 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a528c-70b8-4dc6-bc45-a4ac34c61313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:34.195227 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a528c-70b8-4dc6-bc45-a4ac34c61313 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:34.195502 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a528c-70b8-4dc6-bc45-a4ac34c61313 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:34.195767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a528c-70b8-4dc6-bc45-a4ac34c61313 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:34.206408 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a528c-70b8-4dc6-bc45-a4ac34c61313 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:34.207860 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a528c-70b8-4dc6-bc45-a4ac34c61313 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-719a528c-70b8-4dc6-bc45-a4ac34c61313 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:34.224130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:34.225167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:34.226232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:34.226232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:34.226232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:34.226232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:34.226232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:34.226232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:34.226232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:34.226232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:34.226232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:53:34.226232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:34.233052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 342/684] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:53:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:34.236134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a528c-70b8-4dc6-bc45-a4ac34c61313 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:34.251435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2d543-0983-4195-b854-8edfc6da67bc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:34.290839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fa50c-729e-43cb-b6b1-82301e11ce42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:34.291375 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fa50c-729e-43cb-b6b1-82301e11ce42 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:34.291653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fa50c-729e-43cb-b6b1-82301e11ce42 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:34.291922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fa50c-729e-43cb-b6b1-82301e11ce42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:34.292699 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fa50c-729e-43cb-b6b1-82301e11ce42 None admin] RBAC: Adding query filter params (role.id=035f7d9f445d406eb2e83ca90fc4d5ca, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=d00634cdd3fb41e794d3af0b88ba7338) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:34.293002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fa50c-729e-43cb-b6b1-82301e11ce42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:34.294968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fa50c-729e-43cb-b6b1-82301e11ce42 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:34.304870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 343/685] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:34 2022] GET /identity/v3/role_assignments?user.id=d00634cdd3fb41e794d3af0b88ba7338&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=035f7d9f445d406eb2e83ca90fc4d5ca => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:34.305714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fa50c-729e-43cb-b6b1-82301e11ce42 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:34.326796 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:34.398091 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:34.443741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a528c-70b8-4dc6-bc45-a4ac34c61313 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:34.469531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fa50c-729e-43cb-b6b1-82301e11ce42 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:34.509243 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a43da-0aa8-4e83-816c-b0388019cb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:34.510419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a43da-0aa8-4e83-816c-b0388019cb74 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:34.511046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a43da-0aa8-4e83-816c-b0388019cb74 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:34.511517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a43da-0aa8-4e83-816c-b0388019cb74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:34.524616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a43da-0aa8-4e83-816c-b0388019cb74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:34.525940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67626577-b585-497b-b4e1-bdb50aa08885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:34.526587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67626577-b585-497b-b4e1-bdb50aa08885 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:34.526867 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67626577-b585-497b-b4e1-bdb50aa08885 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:34.527140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67626577-b585-497b-b4e1-bdb50aa08885 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:34.527882 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67626577-b585-497b-b4e1-bdb50aa08885 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:34.528188 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67626577-b585-497b-b4e1-bdb50aa08885 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:34.529631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67626577-b585-497b-b4e1-bdb50aa08885 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-554a43da-0aa8-4e83-816c-b0388019cb74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:34.532132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:34.533060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:34.533940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:34.533940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:34.533940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:34.533940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:34.533940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:34.533940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:34.533940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:34.533940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:34.533940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:34.533940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:34.533940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:34.533940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:34.536729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 344/686] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:53:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:34.551290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a43da-0aa8-4e83-816c-b0388019cb74 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:34.563586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 343/687] 10.222.0.62 () {56 vars in 1135 bytes} [Fri Sep 9 11:53:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:34.570138 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67626577-b585-497b-b4e1-bdb50aa08885 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:34.643775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:34.690990 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a43da-0aa8-4e83-816c-b0388019cb74 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:34.702765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67626577-b585-497b-b4e1-bdb50aa08885 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:34.727172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57095756-d1cd-43fe-8ff3-b511e7b7b4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:34.727781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57095756-d1cd-43fe-8ff3-b511e7b7b4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:34.728053 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57095756-d1cd-43fe-8ff3-b511e7b7b4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:34.728312 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57095756-d1cd-43fe-8ff3-b511e7b7b4c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:34.728962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57095756-d1cd-43fe-8ff3-b511e7b7b4c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:34.729260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57095756-d1cd-43fe-8ff3-b511e7b7b4c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:34.730641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57095756-d1cd-43fe-8ff3-b511e7b7b4c5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:34.743446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 345/688] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:34.747743 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57095756-d1cd-43fe-8ff3-b511e7b7b4c5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:34.748269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1b881-c531-45ad-a251-17b2134f9917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:34.748873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1b881-c531-45ad-a251-17b2134f9917 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:34.749143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1b881-c531-45ad-a251-17b2134f9917 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:34.749404 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1b881-c531-45ad-a251-17b2134f9917 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:34.778596 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:34.882783 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1b881-c531-45ad-a251-17b2134f9917 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:34.886651 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1b881-c531-45ad-a251-17b2134f9917 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:34.933091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57095756-d1cd-43fe-8ff3-b511e7b7b4c5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:34.982282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:34.994532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 344/689] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:53:35.000948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1b881-c531-45ad-a251-17b2134f9917 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:35.003472 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66698b8f-5af5-4d81-8035-58569b70f4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:35.004157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66698b8f-5af5-4d81-8035-58569b70f4e1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:35.005343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66698b8f-5af5-4d81-8035-58569b70f4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:35.005343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66698b8f-5af5-4d81-8035-58569b70f4e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:35.027676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66698b8f-5af5-4d81-8035-58569b70f4e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:35.029496 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66698b8f-5af5-4d81-8035-58569b70f4e1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-66698b8f-5af5-4d81-8035-58569b70f4e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:35.059129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:35.060153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:35.061043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:35.061043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:35.061043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:35.066399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 346/690] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:35.071905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:35.077827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66698b8f-5af5-4d81-8035-58569b70f4e1 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:35.158062 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1b881-c531-45ad-a251-17b2134f9917 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:35.202800 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9afaec-ab1d-4f19-acef-d8a710559ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:35.203591 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9afaec-ab1d-4f19-acef-d8a710559ccd None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:35.203935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9afaec-ab1d-4f19-acef-d8a710559ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:35.204209 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9afaec-ab1d-4f19-acef-d8a710559ccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:35.215263 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9afaec-ab1d-4f19-acef-d8a710559ccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:35.216709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9afaec-ab1d-4f19-acef-d8a710559ccd None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-1d9afaec-ab1d-4f19-acef-d8a710559ccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:35.238642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:35.239605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:35.240593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:35.240593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:35.240593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:35.240593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 345/691] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:35.247355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9afaec-ab1d-4f19-acef-d8a710559ccd None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:35.345845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66698b8f-5af5-4d81-8035-58569b70f4e1 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:35.351120 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:35.356676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9afaec-ab1d-4f19-acef-d8a710559ccd None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:35.393786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0e28b-a45c-48e3-bc85-b11e7b162bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:35.394696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0e28b-a45c-48e3-bc85-b11e7b162bdc None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:35.395120 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0e28b-a45c-48e3-bc85-b11e7b162bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:35.395536 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0e28b-a45c-48e3-bc85-b11e7b162bdc None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:35.396327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0e28b-a45c-48e3-bc85-b11e7b162bdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:35.396774 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0e28b-a45c-48e3-bc85-b11e7b162bdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:35.398381 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0e28b-a45c-48e3-bc85-b11e7b162bdc None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:35.410976 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97606be0-e457-4ed8-a1b8-dcb614139dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:35.411520 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97606be0-e457-4ed8-a1b8-dcb614139dbc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:35.411790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97606be0-e457-4ed8-a1b8-dcb614139dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:35.412049 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97606be0-e457-4ed8-a1b8-dcb614139dbc None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:35.412689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97606be0-e457-4ed8-a1b8-dcb614139dbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:35.413502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97606be0-e457-4ed8-a1b8-dcb614139dbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:35.416626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97606be0-e457-4ed8-a1b8-dcb614139dbc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:35.427738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 346/692] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:35.434153 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0e28b-a45c-48e3-bc85-b11e7b162bdc None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:35.446108 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 347/693] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:35.451657 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97606be0-e457-4ed8-a1b8-dcb614139dbc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:35.533901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:35.581173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0e28b-a45c-48e3-bc85-b11e7b162bdc None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:35.621220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79687eb-ef62-4db0-a4e6-1c2acc2a1142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:35.621928 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79687eb-ef62-4db0-a4e6-1c2acc2a1142 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:35.622296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79687eb-ef62-4db0-a4e6-1c2acc2a1142 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:35.622589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79687eb-ef62-4db0-a4e6-1c2acc2a1142 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:35.630623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79687eb-ef62-4db0-a4e6-1c2acc2a1142 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:35.632426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79687eb-ef62-4db0-a4e6-1c2acc2a1142 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:35.645362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-a79687eb-ef62-4db0-a4e6-1c2acc2a1142 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:35.645362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:35.645362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:35.645362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:35.645362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:35.645362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:35.645362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:35.645362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:35.645362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:35.645362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:35.645362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:35.645362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:35.645362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:35.645362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:35.645362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:35.646386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:35.647866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:35.647866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:35.647866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:35.647866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:35.647866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:35.647866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:35.647866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:35.647866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:35.647866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 347/694] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:35.654833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79687eb-ef62-4db0-a4e6-1c2acc2a1142 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:35.676592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:35.695732 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97606be0-e457-4ed8-a1b8-dcb614139dbc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:35.762306 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837b9c1-3cee-47fd-9d5e-ae0cdc2d3870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:35.763375 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837b9c1-3cee-47fd-9d5e-ae0cdc2d3870 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:35.763862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837b9c1-3cee-47fd-9d5e-ae0cdc2d3870 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:35.764347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837b9c1-3cee-47fd-9d5e-ae0cdc2d3870 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:35.775396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837b9c1-3cee-47fd-9d5e-ae0cdc2d3870 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:35.782776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837b9c1-3cee-47fd-9d5e-ae0cdc2d3870 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-8837b9c1-3cee-47fd-9d5e-ae0cdc2d3870 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:35.801790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:35.803055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:35.804092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:35.804092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:35.804092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:35.804092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:35.804092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:35.804092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:35.804092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:35.804092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:35.804092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:53:35.804092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:35.817836 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 348/695] 10.222.0.62 () {56 vars in 1093 bytes} [Fri Sep 9 11:53:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:35.819171 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837b9c1-3cee-47fd-9d5e-ae0cdc2d3870 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:35.842515 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79687eb-ef62-4db0-a4e6-1c2acc2a1142 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:35.904126 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fbac8-3aa9-4696-9f4d-929c22cc1e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:35.905365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fbac8-3aa9-4696-9f4d-929c22cc1e70 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:35.905873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fbac8-3aa9-4696-9f4d-929c22cc1e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:35.906532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fbac8-3aa9-4696-9f4d-929c22cc1e70 None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:35.907825 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fbac8-3aa9-4696-9f4d-929c22cc1e70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:35.908620 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fbac8-3aa9-4696-9f4d-929c22cc1e70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:35.910812 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fbac8-3aa9-4696-9f4d-929c22cc1e70 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:35.925923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 348/696] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:35.932263 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fbac8-3aa9-4696-9f4d-929c22cc1e70 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:36.010490 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:36.020233 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837b9c1-3cee-47fd-9d5e-ae0cdc2d3870 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:36.066331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:36.090148 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be32ae7-bca9-40a6-bce7-5aae83461cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:36.090787 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be32ae7-bca9-40a6-bce7-5aae83461cff None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:36.091061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be32ae7-bca9-40a6-bce7-5aae83461cff None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:36.091329 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be32ae7-bca9-40a6-bce7-5aae83461cff None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:36.092069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be32ae7-bca9-40a6-bce7-5aae83461cff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:36.092372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be32ae7-bca9-40a6-bce7-5aae83461cff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:36.093803 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be32ae7-bca9-40a6-bce7-5aae83461cff None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:36.119580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fbac8-3aa9-4696-9f4d-929c22cc1e70 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:36.128361 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 349/697] 10.222.0.62 () {56 vars in 1133 bytes} [Fri Sep 9 11:53:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:36.134602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be32ae7-bca9-40a6-bce7-5aae83461cff None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:36.180310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d98247-5166-4ac9-9f75-a1d253c09976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:36.181299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d98247-5166-4ac9-9f75-a1d253c09976 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:36.181575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d98247-5166-4ac9-9f75-a1d253c09976 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:36.181841 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d98247-5166-4ac9-9f75-a1d253c09976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:36.182668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d98247-5166-4ac9-9f75-a1d253c09976 None admin] RBAC: Adding query filter params (role.id=035f7d9f445d406eb2e83ca90fc4d5ca, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=1b2de52919e54bf9bcc352909c9f905f) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:36.182917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d98247-5166-4ac9-9f75-a1d253c09976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:36.184889 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d98247-5166-4ac9-9f75-a1d253c09976 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:36.192848 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 349/698] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:35 2022] GET /identity/v3/role_assignments?user.id=1b2de52919e54bf9bcc352909c9f905f&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=035f7d9f445d406eb2e83ca90fc4d5ca => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:36.198384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025cdf3-9c78-4fd9-b2de-1846eedd5157 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:36.203534 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025cdf3-9c78-4fd9-b2de-1846eedd5157 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:36.203534 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025cdf3-9c78-4fd9-b2de-1846eedd5157 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:36.203998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 350/699] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:36 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:36.210079 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49947f44-5f94-46ba-8822-c314a2c4f7f6 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:36.210411 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49947f44-5f94-46ba-8822-c314a2c4f7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:36.210678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49947f44-5f94-46ba-8822-c314a2c4f7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:36.248200 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-49947f44-5f94-46ba-8822-c314a2c4f7f6 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:36.249436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-49947f44-5f94-46ba-8822-c314a2c4f7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:36.311971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be32ae7-bca9-40a6-bce7-5aae83461cff None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:36.383612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f2b21-57d3-4d40-afe3-08c2a0dab27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:36.384284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2b21-57d3-4d40-afe3-08c2a0dab27c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:36.384569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2b21-57d3-4d40-afe3-08c2a0dab27c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:36.384844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2b21-57d3-4d40-afe3-08c2a0dab27c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:36.392236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 351/700] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:36.421090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:36.473798 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f2b21-57d3-4d40-afe3-08c2a0dab27c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:36.475672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f2b21-57d3-4d40-afe3-08c2a0dab27c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:36.556479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5567bdf-3d53-4126-9991-987e135335ef None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:36.556766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5567bdf-3d53-4126-9991-987e135335ef None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:36.557041 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5567bdf-3d53-4126-9991-987e135335ef None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:36.558704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 352/701] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:53:36.564634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ab545-fadd-408e-9955-334f2a90edf6 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:36.564992 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ab545-fadd-408e-9955-334f2a90edf6 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:36.565275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ab545-fadd-408e-9955-334f2a90edf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:36.597920 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-e18ab545-fadd-408e-9955-334f2a90edf6 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:36.599170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e18ab545-fadd-408e-9955-334f2a90edf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:36.601372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 350/702] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:53:36.608207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f2b21-57d3-4d40-afe3-08c2a0dab27c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:36.681987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:36.685554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:36.692923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 353/703] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:53:36.734447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f2b21-57d3-4d40-afe3-08c2a0dab27c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:36.741879 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ab545-fadd-408e-9955-334f2a90edf6 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:36.773662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb8052-68d7-4f63-b936-4ec77e14ff60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:36.774325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb8052-68d7-4f63-b936-4ec77e14ff60 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:36.774602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb8052-68d7-4f63-b936-4ec77e14ff60 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:36.777134 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb8052-68d7-4f63-b936-4ec77e14ff60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:36.786338 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb8052-68d7-4f63-b936-4ec77e14ff60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:36.787911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb8052-68d7-4f63-b936-4ec77e14ff60 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-17fb8052-68d7-4f63-b936-4ec77e14ff60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:36.805091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:36.806246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:36.807149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:36.807149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:36.807149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:36.810344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 351/704] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:36.811780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb8052-68d7-4f63-b936-4ec77e14ff60 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:36.868690 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ab545-fadd-408e-9955-334f2a90edf6 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:36.915386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec74ffc-e398-4bde-aa79-47eeb4d2fbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:36.916426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec74ffc-e398-4bde-aa79-47eeb4d2fbdd None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:36.916903 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec74ffc-e398-4bde-aa79-47eeb4d2fbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:36.917367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec74ffc-e398-4bde-aa79-47eeb4d2fbdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:36.928968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb8052-68d7-4f63-b936-4ec77e14ff60 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:36.940930 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec74ffc-e398-4bde-aa79-47eeb4d2fbdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:36.941391 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:36.943252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec74ffc-e398-4bde-aa79-47eeb4d2fbdd None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-5ec74ffc-e398-4bde-aa79-47eeb4d2fbdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:36.961701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:36.962818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:36.964213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:36.964213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:36.964213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:36.964213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 354/705] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:36.971164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec74ffc-e398-4bde-aa79-47eeb4d2fbdd None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:36.972204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed7caa-7029-40c6-ba6f-b922c2460c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:36.973027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed7caa-7029-40c6-ba6f-b922c2460c31 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:36.973611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed7caa-7029-40c6-ba6f-b922c2460c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:36.974236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed7caa-7029-40c6-ba6f-b922c2460c31 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:36.975260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed7caa-7029-40c6-ba6f-b922c2460c31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:36.975791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed7caa-7029-40c6-ba6f-b922c2460c31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:36.977542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed7caa-7029-40c6-ba6f-b922c2460c31 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:36.992011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 352/706] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:36.997970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed7caa-7029-40c6-ba6f-b922c2460c31 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:37.078485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:37.119605 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed7caa-7029-40c6-ba6f-b922c2460c31 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:37.148143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec74ffc-e398-4bde-aa79-47eeb4d2fbdd None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:37.161600 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfca88-f41e-44fb-ab31-ca29af15b17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:37.162337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfca88-f41e-44fb-ab31-ca29af15b17b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:37.162662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfca88-f41e-44fb-ab31-ca29af15b17b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:37.162961 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfca88-f41e-44fb-ab31-ca29af15b17b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:37.170523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfca88-f41e-44fb-ab31-ca29af15b17b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:37.171961 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfca88-f41e-44fb-ab31-ca29af15b17b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:37.185512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-a5cfca88-f41e-44fb-ab31-ca29af15b17b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:37.185512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:37.185512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:37.185512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:37.185512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:37.185512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:37.185512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:37.185512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:37.185512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:37.185512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:37.185512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:37.185512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:37.185512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:37.185512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:37.185512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:37.186663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:37.187548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:37.187548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:37.187548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:37.187548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:37.187548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:37.187548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:37.187548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:37.187548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:37.187962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c86c29-55ad-4e28-b962-e1375dc8510e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:37.187962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c86c29-55ad-4e28-b962-e1375dc8510e None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:37.187962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c86c29-55ad-4e28-b962-e1375dc8510e None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:37.187962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c86c29-55ad-4e28-b962-e1375dc8510e None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:37.188262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c86c29-55ad-4e28-b962-e1375dc8510e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:37.188262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c86c29-55ad-4e28-b962-e1375dc8510e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:37.189372 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c86c29-55ad-4e28-b962-e1375dc8510e None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:37.191709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 353/707] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:37.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfca88-f41e-44fb-ab31-ca29af15b17b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:37.203012 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 355/708] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:37.207935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb20422-9d41-49a9-8388-7c00ea87e156 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:37.208203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb20422-9d41-49a9-8388-7c00ea87e156 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:37.209071 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb20422-9d41-49a9-8388-7c00ea87e156 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:37.210740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 356/709] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:37.214937 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb20422-9d41-49a9-8388-7c00ea87e156 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:37.230509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:37.326983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfca88-f41e-44fb-ab31-ca29af15b17b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:37.332483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb20422-9d41-49a9-8388-7c00ea87e156 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:37.347154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:37.366902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8637dd-8d28-4fa7-a408-acd3458bb267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:37.367532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8637dd-8d28-4fa7-a408-acd3458bb267 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:37.367849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8637dd-8d28-4fa7-a408-acd3458bb267 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:37.368084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8637dd-8d28-4fa7-a408-acd3458bb267 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:37.368695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8637dd-8d28-4fa7-a408-acd3458bb267 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:37.370005 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8637dd-8d28-4fa7-a408-acd3458bb267 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:37.373404 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28473a59-cfb0-4a8d-b594-d4e3e395b3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:37.374009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28473a59-cfb0-4a8d-b594-d4e3e395b3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:37.374316 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28473a59-cfb0-4a8d-b594-d4e3e395b3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:37.374589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28473a59-cfb0-4a8d-b594-d4e3e395b3a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:37.375281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28473a59-cfb0-4a8d-b594-d4e3e395b3a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:37.375586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28473a59-cfb0-4a8d-b594-d4e3e395b3a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:37.377503 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28473a59-cfb0-4a8d-b594-d4e3e395b3a2 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-fc8637dd-8d28-4fa7-a408-acd3458bb267 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:37.381380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:37.383093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:37.383930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:37.383930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:37.383930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:37.383930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:37.383930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:37.383930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:37.383930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:37.383930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:37.383930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:37.383930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:37.383930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:53:37.383930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:37.383930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:53:37.383930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:37.383930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 354/710] 10.222.0.62 () {56 vars in 1129 bytes} [Fri Sep 9 11:53:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:37.385419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8637dd-8d28-4fa7-a408-acd3458bb267 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:37.401320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 357/711] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:37.401320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a75d74-7c71-404b-a9af-bb72462e9ad7 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:37.401320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a75d74-7c71-404b-a9af-bb72462e9ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:37.401320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a75d74-7c71-404b-a9af-bb72462e9ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:37.417706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-98a75d74-7c71-404b-a9af-bb72462e9ad7 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:37.419064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-98a75d74-7c71-404b-a9af-bb72462e9ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:37.491503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:37.493265 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 358/712] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:37.494310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a75d74-7c71-404b-a9af-bb72462e9ad7 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:37.504346 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8637dd-8d28-4fa7-a408-acd3458bb267 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:37.542945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947af5c-b373-4974-97b2-9edb7b460b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:37.543749 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947af5c-b373-4974-97b2-9edb7b460b43 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:37.544127 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947af5c-b373-4974-97b2-9edb7b460b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:37.544496 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947af5c-b373-4974-97b2-9edb7b460b43 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:37.545247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947af5c-b373-4974-97b2-9edb7b460b43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:37.547044 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947af5c-b373-4974-97b2-9edb7b460b43 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:37.573278 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 355/713] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:53:37 2022] POST /identity/v3/users => generated 254 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:37.574369 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947af5c-b373-4974-97b2-9edb7b460b43 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:37.646662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a75d74-7c71-404b-a9af-bb72462e9ad7 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:37.649484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:37.701940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0eee64-eb58-4d5e-a88a-48fd2559f9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:37.703684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eee64-eb58-4d5e-a88a-48fd2559f9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:37.704168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eee64-eb58-4d5e-a88a-48fd2559f9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:37.704867 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eee64-eb58-4d5e-a88a-48fd2559f9c9 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:37.705740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0eee64-eb58-4d5e-a88a-48fd2559f9c9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:37.706519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0eee64-eb58-4d5e-a88a-48fd2559f9c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:37.709028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0eee64-eb58-4d5e-a88a-48fd2559f9c9 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:37.715037 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947af5c-b373-4974-97b2-9edb7b460b43 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:37.716655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 359/714] 10.222.0.62 () {56 vars in 1124 bytes} [Fri Sep 9 11:53:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:37.724588 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0eee64-eb58-4d5e-a88a-48fd2559f9c9 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:37.760397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35751f-5729-4ca7-8f8d-70351493331e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:37.760651 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35751f-5729-4ca7-8f8d-70351493331e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:37.760955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35751f-5729-4ca7-8f8d-70351493331e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:37.761303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35751f-5729-4ca7-8f8d-70351493331e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:37.762110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35751f-5729-4ca7-8f8d-70351493331e None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=1b30e80c421245b5a1ed5849b28ea90e) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:37.762615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35751f-5729-4ca7-8f8d-70351493331e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:37.763998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35751f-5729-4ca7-8f8d-70351493331e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:37.771323 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 356/715] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:37 2022] GET /identity/v3/role_assignments?user.id=1b30e80c421245b5a1ed5849b28ea90e&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:37.875044 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35751f-5729-4ca7-8f8d-70351493331e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:37.907627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:37.920239 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0eee64-eb58-4d5e-a88a-48fd2559f9c9 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:37.970875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102596d-0e01-414c-a3fa-fe84a7f94245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:37.972130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102596d-0e01-414c-a3fa-fe84a7f94245 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:37.973089 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102596d-0e01-414c-a3fa-fe84a7f94245 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:37.973854 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102596d-0e01-414c-a3fa-fe84a7f94245 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:37.974773 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102596d-0e01-414c-a3fa-fe84a7f94245 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:37.975566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102596d-0e01-414c-a3fa-fe84a7f94245 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:37.977733 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102596d-0e01-414c-a3fa-fe84a7f94245 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:37.986807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 360/716] 10.222.0.62 () {56 vars in 1124 bytes} [Fri Sep 9 11:53:37 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:38.034715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35751f-5729-4ca7-8f8d-70351493331e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:38.079689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68544f-60ec-49fa-82c9-fa33a3876b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:38.080342 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68544f-60ec-49fa-82c9-fa33a3876b27 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:38.080621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68544f-60ec-49fa-82c9-fa33a3876b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:38.080887 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68544f-60ec-49fa-82c9-fa33a3876b27 None admin] PATH_INFO: `/v3/services/network` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:38.081473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68544f-60ec-49fa-82c9-fa33a3876b27 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:38.082968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68544f-60ec-49fa-82c9-fa33a3876b27 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-5c68544f-60ec-49fa-82c9-fa33a3876b27 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:38.092948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:38.094015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:38.095530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:38.095530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:38.095530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:38.095530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:38.095530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:38.095530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:38.095530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:38.095530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:38.095530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:38.095530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:38.095530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:53:38.095530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:38.095530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:53:38.095530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:38.095530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 357/717] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:37 2022] GET /identity/v3/services/network => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:38.103289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102596d-0e01-414c-a3fa-fe84a7f94245 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:38.167251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:38.262300 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102596d-0e01-414c-a3fa-fe84a7f94245 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:38.304430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b86db-5c9f-4e71-9bf8-40ff2783c694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:38.305065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b86db-5c9f-4e71-9bf8-40ff2783c694 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:38.305347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b86db-5c9f-4e71-9bf8-40ff2783c694 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:38.305621 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b86db-5c9f-4e71-9bf8-40ff2783c694 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:38.306283 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b86db-5c9f-4e71-9bf8-40ff2783c694 None admin] RBAC: Adding query filter params (name=network) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:38.306590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b86db-5c9f-4e71-9bf8-40ff2783c694 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:38.307975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b86db-5c9f-4e71-9bf8-40ff2783c694 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:38.313011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 361/718] 10.222.0.62 () {56 vars in 1112 bytes} [Fri Sep 9 11:53:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:38.318325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68544f-60ec-49fa-82c9-fa33a3876b27 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:38.445837 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:38.475978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68544f-60ec-49fa-82c9-fa33a3876b27 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:38.527163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba47055-7100-4e0b-9e88-0d1908b05774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:38.527774 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba47055-7100-4e0b-9e88-0d1908b05774 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:38.528048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba47055-7100-4e0b-9e88-0d1908b05774 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:38.528313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba47055-7100-4e0b-9e88-0d1908b05774 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:38.528944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba47055-7100-4e0b-9e88-0d1908b05774 None admin] RBAC: Adding query filter params (type=network) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:38.529239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba47055-7100-4e0b-9e88-0d1908b05774 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:38.530617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba47055-7100-4e0b-9e88-0d1908b05774 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:38.540418 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 358/719] 10.222.0.62 () {56 vars in 1112 bytes} [Fri Sep 9 11:53:38 2022] GET /identity/v3/services?type=network => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:38.979898 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517de00-1c40-402b-a752-d258e074d729 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:38.980329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517de00-1c40-402b-a752-d258e074d729 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:38.980642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517de00-1c40-402b-a752-d258e074d729 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:38.982449 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 362/720] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:38.988662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc8d0f-20f6-4710-b472-829d564dc420 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:38.988942 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc8d0f-20f6-4710-b472-829d564dc420 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:38.989215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc8d0f-20f6-4710-b472-829d564dc420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:39.009854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1edc8d0f-20f6-4710-b472-829d564dc420 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:39.011412 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1edc8d0f-20f6-4710-b472-829d564dc420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:39.107948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 359/721] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:39.518775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517de00-1c40-402b-a752-d258e074d729 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:39.647929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:39.655450 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517de00-1c40-402b-a752-d258e074d729 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:39.710880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892744e-bd86-411a-adc5-bada0a343490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:39.711526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892744e-bd86-411a-adc5-bada0a343490 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:39.711803 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892744e-bd86-411a-adc5-bada0a343490 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:39.712072 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892744e-bd86-411a-adc5-bada0a343490 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:39.729611 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892744e-bd86-411a-adc5-bada0a343490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-8892744e-bd86-411a-adc5-bada0a343490 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:39.734691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:39.735652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:39.736538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:39.736538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:39.736538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:39.736538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:39.736538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:39.736538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:39.736538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:39.736538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:39.736538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:39.736538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:39.736538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:39.736538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:39.736538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 363/722] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:53:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:39.742461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc8d0f-20f6-4710-b472-829d564dc420 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:39.830598 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:39.863152 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc8d0f-20f6-4710-b472-829d564dc420 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:39.917727 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de94767-aac8-40bf-9b87-41cb1722dce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:39.918399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de94767-aac8-40bf-9b87-41cb1722dce3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:39.918679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de94767-aac8-40bf-9b87-41cb1722dce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:39.918951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de94767-aac8-40bf-9b87-41cb1722dce3 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:39.919644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de94767-aac8-40bf-9b87-41cb1722dce3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:39.919948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de94767-aac8-40bf-9b87-41cb1722dce3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:39.921344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de94767-aac8-40bf-9b87-41cb1722dce3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:39.931738 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 360/723] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:39.938032 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892744e-bd86-411a-adc5-bada0a343490 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:40.095222 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892744e-bd86-411a-adc5-bada0a343490 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:40.140717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee713cf1-af9a-4ba6-9a76-251c6e13ac10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:40.141380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee713cf1-af9a-4ba6-9a76-251c6e13ac10 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:40.141657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee713cf1-af9a-4ba6-9a76-251c6e13ac10 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:40.141925 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee713cf1-af9a-4ba6-9a76-251c6e13ac10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:40.157936 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee713cf1-af9a-4ba6-9a76-251c6e13ac10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:40.159463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee713cf1-af9a-4ba6-9a76-251c6e13ac10 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-ee713cf1-af9a-4ba6-9a76-251c6e13ac10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:40.179559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:40.181827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:40.183243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:40.183243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:40.183243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:40.183243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 364/724] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:40.186417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de94767-aac8-40bf-9b87-41cb1722dce3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:40.304330 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de94767-aac8-40bf-9b87-41cb1722dce3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:40.339647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d963c3b-9359-4c59-99c7-879a186b778e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:40.340332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d963c3b-9359-4c59-99c7-879a186b778e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:40.340626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d963c3b-9359-4c59-99c7-879a186b778e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:40.340920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d963c3b-9359-4c59-99c7-879a186b778e None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:40.341570 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d963c3b-9359-4c59-99c7-879a186b778e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:40.341868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d963c3b-9359-4c59-99c7-879a186b778e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:40.343303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d963c3b-9359-4c59-99c7-879a186b778e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:40.356552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:40.357019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 361/725] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:40.364579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee713cf1-af9a-4ba6-9a76-251c6e13ac10 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:40.386845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:40.522705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee713cf1-af9a-4ba6-9a76-251c6e13ac10 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:40.532850 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677c857-bd79-4f8c-a115-b0251dde74ae None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:40.533140 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677c857-bd79-4f8c-a115-b0251dde74ae None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:40.533404 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677c857-bd79-4f8c-a115-b0251dde74ae None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:40.535009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 362/726] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:40.543855 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404cc27-d04f-48ee-b0b0-62bd8f1deeb4 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:40.544192 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404cc27-d04f-48ee-b0b0-62bd8f1deeb4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:40.544471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404cc27-d04f-48ee-b0b0-62bd8f1deeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:40.567373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b404cc27-d04f-48ee-b0b0-62bd8f1deeb4 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:40.568570 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b404cc27-d04f-48ee-b0b0-62bd8f1deeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:40.584116 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e2fc6-0790-49ec-ab87-8321b9762fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:40.584770 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e2fc6-0790-49ec-ab87-8321b9762fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:40.585041 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e2fc6-0790-49ec-ab87-8321b9762fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:40.585306 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e2fc6-0790-49ec-ab87-8321b9762fe5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:40.594738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e2fc6-0790-49ec-ab87-8321b9762fe5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:40.596166 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e2fc6-0790-49ec-ab87-8321b9762fe5 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-dd4e2fc6-0790-49ec-ab87-8321b9762fe5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:40.616680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:40.618968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:40.621133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:40.621133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:40.621133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:40.621133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:40.621133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:40.621133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:40.621133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:40.621133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:40.621133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:53:40.621133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:40.621133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 365/727] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:53:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:40.626315 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e2fc6-0790-49ec-ab87-8321b9762fe5 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:40.687045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 363/728] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:40.739668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:40.766656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bdd48-627e-4663-babf-5a4c85b49898 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:40.767174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bdd48-627e-4663-babf-5a4c85b49898 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:40.767252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bdd48-627e-4663-babf-5a4c85b49898 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:40.768580 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 364/729] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:53:40.775213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1e588-9196-45d4-9a22-9dcff2b1a0e8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:40.775814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1e588-9196-45d4-9a22-9dcff2b1a0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:40.776265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1e588-9196-45d4-9a22-9dcff2b1a0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:40.801027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e2fc6-0790-49ec-ab87-8321b9762fe5 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:40.808522 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-79f1e588-9196-45d4-9a22-9dcff2b1a0e8 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:40.810029 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-79f1e588-9196-45d4-9a22-9dcff2b1a0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:40.840778 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af433aec-ccc8-46a9-b772-98aa41f5652c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:40.841406 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af433aec-ccc8-46a9-b772-98aa41f5652c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:40.841684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af433aec-ccc8-46a9-b772-98aa41f5652c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:40.841950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af433aec-ccc8-46a9-b772-98aa41f5652c None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:40.842733 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af433aec-ccc8-46a9-b772-98aa41f5652c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:40.843038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af433aec-ccc8-46a9-b772-98aa41f5652c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:40.844524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af433aec-ccc8-46a9-b772-98aa41f5652c None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:40.889611 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 366/730] 10.222.0.62 () {56 vars in 1135 bytes} [Fri Sep 9 11:53:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:40.899003 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af433aec-ccc8-46a9-b772-98aa41f5652c None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:40.945103 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 365/731] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:53:40.949781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00949ed-d214-4327-b0e4-826ed8b02ebe None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:40.950059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00949ed-d214-4327-b0e4-826ed8b02ebe None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:40.950360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00949ed-d214-4327-b0e4-826ed8b02ebe None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:40.951934 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 366/732] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:53:40.957696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce1183-9cf4-4b89-8bcf-1b6f0a95acce None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:40.958187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce1183-9cf4-4b89-8bcf-1b6f0a95acce None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:40.958613 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce1183-9cf4-4b89-8bcf-1b6f0a95acce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:40.990370 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5cce1183-9cf4-4b89-8bcf-1b6f0a95acce None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:40.993287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5cce1183-9cf4-4b89-8bcf-1b6f0a95acce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:41.048624 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af433aec-ccc8-46a9-b772-98aa41f5652c None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:41.062370 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:41.076036 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:41.088791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec947765-ae19-45d3-889c-bf46271f9257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:41.089393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec947765-ae19-45d3-889c-bf46271f9257 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:41.089667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec947765-ae19-45d3-889c-bf46271f9257 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:41.089936 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec947765-ae19-45d3-889c-bf46271f9257 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:41.094892 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 367/733] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:53:41.100371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce1183-9cf4-4b89-8bcf-1b6f0a95acce None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:41.216952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec947765-ae19-45d3-889c-bf46271f9257 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:41.218363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec947765-ae19-45d3-889c-bf46271f9257 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:41.254828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce1183-9cf4-4b89-8bcf-1b6f0a95acce None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:41.300691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ece25-c0a8-41e6-ae75-94003656ac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:41.301351 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ece25-c0a8-41e6-ae75-94003656ac27 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:41.301618 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ece25-c0a8-41e6-ae75-94003656ac27 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:41.301899 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ece25-c0a8-41e6-ae75-94003656ac27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:41.315626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ece25-c0a8-41e6-ae75-94003656ac27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-725ece25-c0a8-41e6-ae75-94003656ac27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:41.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:41.322062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:41.322970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:41.322970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:41.322970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:41.322970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:41.322970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:41.322970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:41.322970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:41.322970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:41.322970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:41.322970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:41.322970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:41.322970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:41.327162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 368/734] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:53:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:41.328014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ece25-c0a8-41e6-ae75-94003656ac27 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:41.356885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 367/735] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:53:41.358412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec947765-ae19-45d3-889c-bf46271f9257 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:41.379748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:41.397843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:41.449200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ece25-c0a8-41e6-ae75-94003656ac27 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:41.480502 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec947765-ae19-45d3-889c-bf46271f9257 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:41.483543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c34fc5-d90d-49a9-b74b-bf8a6216db8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:41.484396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c34fc5-d90d-49a9-b74b-bf8a6216db8f None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:41.484672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c34fc5-d90d-49a9-b74b-bf8a6216db8f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:41.484939 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c34fc5-d90d-49a9-b74b-bf8a6216db8f None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:41.485604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c34fc5-d90d-49a9-b74b-bf8a6216db8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:41.486973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c34fc5-d90d-49a9-b74b-bf8a6216db8f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:41.503146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 369/736] 10.222.0.62 () {60 vars in 1141 bytes} [Fri Sep 9 11:53:41 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:41.504130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c34fc5-d90d-49a9-b74b-bf8a6216db8f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:41.527406 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da930b-6761-423c-a4ab-64d25dadc81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:41.528138 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da930b-6761-423c-a4ab-64d25dadc81a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:41.528418 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da930b-6761-423c-a4ab-64d25dadc81a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:41.528682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da930b-6761-423c-a4ab-64d25dadc81a None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:41.529366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da930b-6761-423c-a4ab-64d25dadc81a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:41.529668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da930b-6761-423c-a4ab-64d25dadc81a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:41.531232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da930b-6761-423c-a4ab-64d25dadc81a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:41.542581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 368/737] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:41.543473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da930b-6761-423c-a4ab-64d25dadc81a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:41.635968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c34fc5-d90d-49a9-b74b-bf8a6216db8f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:41.636220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:41.638884 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:41.661499 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da930b-6761-423c-a4ab-64d25dadc81a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:41.676342 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4daedc-0dce-4a1e-a836-eb6ab8af95f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:41.676986 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4daedc-0dce-4a1e-a836-eb6ab8af95f6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:41.677260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4daedc-0dce-4a1e-a836-eb6ab8af95f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:41.677533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4daedc-0dce-4a1e-a836-eb6ab8af95f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:41.691359 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4daedc-0dce-4a1e-a836-eb6ab8af95f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:41.692805 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4daedc-0dce-4a1e-a836-eb6ab8af95f6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:41.703262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c69100-4e45-40f5-bc4d-b3ab01a87965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:41.703994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c69100-4e45-40f5-bc4d-b3ab01a87965 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:41.704177 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c69100-4e45-40f5-bc4d-b3ab01a87965 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:41.704447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c69100-4e45-40f5-bc4d-b3ab01a87965 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-7d4daedc-0dce-4a1e-a836-eb6ab8af95f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:41.711566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:41.712545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:41.713449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:41.713449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:41.713449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:41.713449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 370/738] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:41.714085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432de5e8-c2bf-474a-af33-c1293ba8e25c None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:41.714305 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432de5e8-c2bf-474a-af33-c1293ba8e25c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:41.714605 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432de5e8-c2bf-474a-af33-c1293ba8e25c None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:41.715975 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 371/739] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:41.716924 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432de5e8-c2bf-474a-af33-c1293ba8e25c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:41.719601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c69100-4e45-40f5-bc4d-b3ab01a87965 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-12c69100-4e45-40f5-bc4d-b3ab01a87965 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:41.724994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:41.725941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:41.727169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:41.727169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:41.727169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:41.727169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:41.727169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:41.727169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:41.727169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:41.727169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:41.727169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:41.727169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:41.727169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:41.727169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:41.727169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 369/740] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:41.727169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c69100-4e45-40f5-bc4d-b3ab01a87965 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:41.831270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432de5e8-c2bf-474a-af33-c1293ba8e25c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:41.840833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c69100-4e45-40f5-bc4d-b3ab01a87965 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:41.869062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619c94c-a488-4070-9612-7dbbe0011e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:41.869702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619c94c-a488-4070-9612-7dbbe0011e45 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:41.869980 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619c94c-a488-4070-9612-7dbbe0011e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:41.870286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619c94c-a488-4070-9612-7dbbe0011e45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:41.876512 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb7168-c5a2-43b4-b5e6-6243f69ca333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:41.877217 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb7168-c5a2-43b4-b5e6-6243f69ca333 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:41.877488 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb7168-c5a2-43b4-b5e6-6243f69ca333 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:41.877750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb7168-c5a2-43b4-b5e6-6243f69ca333 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:41.878807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb7168-c5a2-43b4-b5e6-6243f69ca333 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:41.878807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb7168-c5a2-43b4-b5e6-6243f69ca333 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:41.880187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb7168-c5a2-43b4-b5e6-6243f69ca333 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:41.881104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619c94c-a488-4070-9612-7dbbe0011e45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:41.883189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619c94c-a488-4070-9612-7dbbe0011e45 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:41.896743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 370/741] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:41.898398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e11eab-f606-436f-8882-e0093edf8052 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:41.898838 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e11eab-f606-436f-8882-e0093edf8052 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:41.899266 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e11eab-f606-436f-8882-e0093edf8052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-0619c94c-a488-4070-9612-7dbbe0011e45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:41.900271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:41.901191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:41.902068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:41.902068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:41.902068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:41.902068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 372/742] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:41.902344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619c94c-a488-4070-9612-7dbbe0011e45 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:41.909736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:41.915372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:41.924764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-26e11eab-f606-436f-8882-e0093edf8052 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:41.925963 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-26e11eab-f606-436f-8882-e0093edf8052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:41.997835 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 371/743] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:41 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:53:41.998949 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e11eab-f606-436f-8882-e0093edf8052 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:42.030652 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619c94c-a488-4070-9612-7dbbe0011e45 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:42.067654 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e162c-1fde-4b2a-9cd2-e1668a640fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:42.068288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e162c-1fde-4b2a-9cd2-e1668a640fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:42.068569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e162c-1fde-4b2a-9cd2-e1668a640fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:42.068946 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e162c-1fde-4b2a-9cd2-e1668a640fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:42.069618 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e162c-1fde-4b2a-9cd2-e1668a640fa7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:42.069921 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e162c-1fde-4b2a-9cd2-e1668a640fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:42.071327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e162c-1fde-4b2a-9cd2-e1668a640fa7 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:42.082459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 373/744] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:42.083422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e162c-1fde-4b2a-9cd2-e1668a640fa7 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:42.133141 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e11eab-f606-436f-8882-e0093edf8052 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:42.164532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:42.170302 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0b9ff-5384-4fa3-b173-ce8659ad0c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:42.171042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0b9ff-5384-4fa3-b173-ce8659ad0c93 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:42.171372 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0b9ff-5384-4fa3-b173-ce8659ad0c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:42.171650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0b9ff-5384-4fa3-b173-ce8659ad0c93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:42.180700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:42.183045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0b9ff-5384-4fa3-b173-ce8659ad0c93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:42.185300 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0b9ff-5384-4fa3-b173-ce8659ad0c93 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:42.202059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-72d0b9ff-5384-4fa3-b173-ce8659ad0c93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:42.202059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:42.202059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:42.202059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:42.202059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:42.202059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:42.202059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:42.202059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:42.202059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:42.202059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:42.202059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:42.202059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:42.202059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:42.202059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:42.202059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:42.203034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:42.203935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:42.203935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:42.203935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:42.203935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:42.203935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:42.203935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:42.203935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:42.203935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:42.206323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 372/745] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:42.207326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0b9ff-5384-4fa3-b173-ce8659ad0c93 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:42.221554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e162c-1fde-4b2a-9cd2-e1668a640fa7 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:42.263393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af258a8c-683c-4603-a9d8-7dc01c600aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:42.264074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af258a8c-683c-4603-a9d8-7dc01c600aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:42.264353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af258a8c-683c-4603-a9d8-7dc01c600aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:42.264646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af258a8c-683c-4603-a9d8-7dc01c600aa1 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:42.265320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af258a8c-683c-4603-a9d8-7dc01c600aa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:42.265625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af258a8c-683c-4603-a9d8-7dc01c600aa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:42.267064 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af258a8c-683c-4603-a9d8-7dc01c600aa1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:42.280577 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 374/746] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:42.281488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af258a8c-683c-4603-a9d8-7dc01c600aa1 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:42.317555 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0b9ff-5384-4fa3-b173-ce8659ad0c93 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:42.353841 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d4302-808f-4369-bed2-75be4fc593bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:42.354434 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d4302-808f-4369-bed2-75be4fc593bf None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:42.354713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d4302-808f-4369-bed2-75be4fc593bf None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:42.355009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d4302-808f-4369-bed2-75be4fc593bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:42.366766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d4302-808f-4369-bed2-75be4fc593bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:42.368212 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d4302-808f-4369-bed2-75be4fc593bf None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-173d4302-808f-4369-bed2-75be4fc593bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:42.383680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:42.384693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:42.385663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:42.385663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:42.385663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:42.385663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 373/747] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:42.385663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d4302-808f-4369-bed2-75be4fc593bf None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:42.397641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af258a8c-683c-4603-a9d8-7dc01c600aa1 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:42.431161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:42.439308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c039c2-4626-45b3-a193-2caa057473ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:42.439765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:42.439924 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c039c2-4626-45b3-a193-2caa057473ec None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:42.440194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c039c2-4626-45b3-a193-2caa057473ec None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:42.440460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c039c2-4626-45b3-a193-2caa057473ec None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:42.441159 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c039c2-4626-45b3-a193-2caa057473ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:42.441464 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c039c2-4626-45b3-a193-2caa057473ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:42.442919 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c039c2-4626-45b3-a193-2caa057473ec None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:42.460674 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 375/748] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:42.460674 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c039c2-4626-45b3-a193-2caa057473ec None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:42.519867 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d4302-808f-4369-bed2-75be4fc593bf None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:42.555891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f8e5d-95e0-4738-a267-0e4f28912339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:42.556543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f8e5d-95e0-4738-a267-0e4f28912339 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:42.556782 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f8e5d-95e0-4738-a267-0e4f28912339 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:42.557059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f8e5d-95e0-4738-a267-0e4f28912339 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:42.566193 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f8e5d-95e0-4738-a267-0e4f28912339 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:42.566676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c039c2-4626-45b3-a193-2caa057473ec None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:42.567666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f8e5d-95e0-4738-a267-0e4f28912339 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-660f8e5d-95e0-4738-a267-0e4f28912339 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:42.582737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:42.583785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:42.585309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:42.585309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:42.585309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:42.585309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:42.585309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:42.585309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:42.585309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:42.585309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:42.585309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:53:42.585309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:42.585309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 374/749] 10.222.0.62 () {56 vars in 1093 bytes} [Fri Sep 9 11:53:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:42.585309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f8e5d-95e0-4738-a267-0e4f28912339 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:42.611257 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb869e-d49e-4743-95fb-94e0df0a0d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:42.611774 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb869e-d49e-4743-95fb-94e0df0a0d97 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:42.612052 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb869e-d49e-4743-95fb-94e0df0a0d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:42.612326 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb869e-d49e-4743-95fb-94e0df0a0d97 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:42.612970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb869e-d49e-4743-95fb-94e0df0a0d97 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:42.614361 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb869e-d49e-4743-95fb-94e0df0a0d97 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:42.629579 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 376/750] 10.222.0.62 () {60 vars in 1141 bytes} [Fri Sep 9 11:53:42 2022] POST /identity/v3/services => generated 240 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:42.630432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb869e-d49e-4743-95fb-94e0df0a0d97 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:42.698534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:42.726687 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f8e5d-95e0-4738-a267-0e4f28912339 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:42.729679 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:42.776223 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb869e-d49e-4743-95fb-94e0df0a0d97 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:42.776900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7238c59-0236-49dd-9370-58cea0cfaea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:42.777779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7238c59-0236-49dd-9370-58cea0cfaea3 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:42.778318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7238c59-0236-49dd-9370-58cea0cfaea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:42.778804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7238c59-0236-49dd-9370-58cea0cfaea3 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:42.779707 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7238c59-0236-49dd-9370-58cea0cfaea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:42.780259 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7238c59-0236-49dd-9370-58cea0cfaea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:42.781880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7238c59-0236-49dd-9370-58cea0cfaea3 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:42.801393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 375/751] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:42.802715 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7238c59-0236-49dd-9370-58cea0cfaea3 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:42.825294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2f29f-21d4-4659-abdf-637812d55254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:42.825918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2f29f-21d4-4659-abdf-637812d55254 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:42.826221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2f29f-21d4-4659-abdf-637812d55254 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:42.826494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2f29f-21d4-4659-abdf-637812d55254 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:42.827228 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2f29f-21d4-4659-abdf-637812d55254 None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=1b2de52919e54bf9bcc352909c9f905f) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:42.827532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2f29f-21d4-4659-abdf-637812d55254 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:42.829450 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2f29f-21d4-4659-abdf-637812d55254 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:42.837662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 377/752] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:42 2022] GET /identity/v3/role_assignments?user.id=1b2de52919e54bf9bcc352909c9f905f&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:42.838288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2f29f-21d4-4659-abdf-637812d55254 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:42.951792 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7238c59-0236-49dd-9370-58cea0cfaea3 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:43.015327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2f29f-21d4-4659-abdf-637812d55254 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:43.054517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:43.057630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d63b21-3a12-43cd-a146-4d41a38c31a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:43.059363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d63b21-3a12-43cd-a146-4d41a38c31a0 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:43.060579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d63b21-3a12-43cd-a146-4d41a38c31a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:43.061287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82aeb6-a5e9-49a2-bb19-c2c429dbf969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:43.061645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d63b21-3a12-43cd-a146-4d41a38c31a0 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:43.062525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d63b21-3a12-43cd-a146-4d41a38c31a0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:43.063218 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82aeb6-a5e9-49a2-bb19-c2c429dbf969 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:43.063591 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d63b21-3a12-43cd-a146-4d41a38c31a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:43.063929 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82aeb6-a5e9-49a2-bb19-c2c429dbf969 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:43.064260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82aeb6-a5e9-49a2-bb19-c2c429dbf969 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:43.065757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d63b21-3a12-43cd-a146-4d41a38c31a0 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:43.076717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82aeb6-a5e9-49a2-bb19-c2c429dbf969 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:43.081771 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82aeb6-a5e9-49a2-bb19-c2c429dbf969 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-8d82aeb6-a5e9-49a2-bb19-c2c429dbf969 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:43.107542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:43.109034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:43.110227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:43.110227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:43.110227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:43.110227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:43.110227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:43.110227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:43.110227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:43.110227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:43.110227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:53:43.110227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:43.118228 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 376/753] 10.222.0.62 () {56 vars in 1133 bytes} [Fri Sep 9 11:53:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:43.119685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 378/754] 10.222.0.62 () {56 vars in 1117 bytes} [Fri Sep 9 11:53:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:43.121063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d63b21-3a12-43cd-a146-4d41a38c31a0 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:43.126623 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82aeb6-a5e9-49a2-bb19-c2c429dbf969 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:43.166761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:43.257106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82aeb6-a5e9-49a2-bb19-c2c429dbf969 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:43.301046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90638dc0-cecd-4e24-aaf7-48abdda19a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:43.301983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90638dc0-cecd-4e24-aaf7-48abdda19a65 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:43.302421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90638dc0-cecd-4e24-aaf7-48abdda19a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:43.302758 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90638dc0-cecd-4e24-aaf7-48abdda19a65 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:43.303945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90638dc0-cecd-4e24-aaf7-48abdda19a65 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:43.304336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90638dc0-cecd-4e24-aaf7-48abdda19a65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:43.306518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90638dc0-cecd-4e24-aaf7-48abdda19a65 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:43.324819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d63b21-3a12-43cd-a146-4d41a38c31a0 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:43.327030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 379/755] 10.222.0.62 () {56 vars in 1149 bytes} [Fri Sep 9 11:53:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:43.333609 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90638dc0-cecd-4e24-aaf7-48abdda19a65 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:43.368661 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:43.407809 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b1238-ee99-4062-9b3b-5ccab1256c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:43.408550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b1238-ee99-4062-9b3b-5ccab1256c66 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:43.409519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b1238-ee99-4062-9b3b-5ccab1256c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:43.410025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b1238-ee99-4062-9b3b-5ccab1256c66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:43.435525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b1238-ee99-4062-9b3b-5ccab1256c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:43.437109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b1238-ee99-4062-9b3b-5ccab1256c66 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-cc8b1238-ee99-4062-9b3b-5ccab1256c66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:43.461610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:43.463187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:43.464281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:43.464281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:43.464281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:43.464281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 377/756] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:43.470594 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b1238-ee99-4062-9b3b-5ccab1256c66 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:43.489962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90638dc0-cecd-4e24-aaf7-48abdda19a65 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:43.560170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1d827-5824-4f0d-93df-aea1c3b1a3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:43.560792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1d827-5824-4f0d-93df-aea1c3b1a3ac None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:43.561071 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1d827-5824-4f0d-93df-aea1c3b1a3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:43.561344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1d827-5824-4f0d-93df-aea1c3b1a3ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:43.634658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:43.686819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b1238-ee99-4062-9b3b-5ccab1256c66 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:43.728690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1d827-5824-4f0d-93df-aea1c3b1a3ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:43.730659 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1d827-5824-4f0d-93df-aea1c3b1a3ac None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:43.742426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09f871-6958-4d3d-9449-8d4981edb897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:43.742976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09f871-6958-4d3d-9449-8d4981edb897 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:43.743255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09f871-6958-4d3d-9449-8d4981edb897 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:43.743526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09f871-6958-4d3d-9449-8d4981edb897 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:43.744187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09f871-6958-4d3d-9449-8d4981edb897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:43.744491 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09f871-6958-4d3d-9449-8d4981edb897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:43.745891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09f871-6958-4d3d-9449-8d4981edb897 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:43.772124 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 378/757] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:43.777932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09f871-6958-4d3d-9449-8d4981edb897 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:43.781954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:43.961849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09f871-6958-4d3d-9449-8d4981edb897 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:44.000404 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa016635-7f7c-4d02-a98b-a673e47842b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:44.001389 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa016635-7f7c-4d02-a98b-a673e47842b4 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:44.001879 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa016635-7f7c-4d02-a98b-a673e47842b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:44.002386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa016635-7f7c-4d02-a98b-a673e47842b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:44.004736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:44.011941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa016635-7f7c-4d02-a98b-a673e47842b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:44.013714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa016635-7f7c-4d02-a98b-a673e47842b4 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:44.026663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-fa016635-7f7c-4d02-a98b-a673e47842b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:44.026663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:44.026663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:44.026663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:44.026663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:44.026663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:44.026663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:44.026663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:44.026663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:44.026663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:44.026663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:44.026663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:44.026663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:44.026663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:44.026663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:44.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:44.028513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:44.028513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:44.028513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:44.028513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:44.028513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:44.028513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:44.028513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:44.028513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:44.035430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 379/758] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:44.037168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28b04c-ebee-4131-9f63-7a9242b292c7 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:44.037736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28b04c-ebee-4131-9f63-7a9242b292c7 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:44.038301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28b04c-ebee-4131-9f63-7a9242b292c7 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:44.040101 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 380/759] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:44.041074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28b04c-ebee-4131-9f63-7a9242b292c7 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:44.052751 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 380/760] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 11:53:44.054668 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bb656-e479-4271-a9a5-b601487bf151 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:44.054954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bb656-e479-4271-a9a5-b601487bf151 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:44.055216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bb656-e479-4271-a9a5-b601487bf151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:44.082398 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9e5bb656-e479-4271-a9a5-b601487bf151 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:44.085027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9e5bb656-e479-4271-a9a5-b601487bf151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:44.159281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28b04c-ebee-4131-9f63-7a9242b292c7 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:44.168979 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 381/761] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:53:44.170095 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bb656-e479-4271-a9a5-b601487bf151 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:44.202803 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076029ad-f20c-4aab-89ff-8b59f03e17ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:44.203846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076029ad-f20c-4aab-89ff-8b59f03e17ba None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:44.204321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076029ad-f20c-4aab-89ff-8b59f03e17ba None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:44.204793 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076029ad-f20c-4aab-89ff-8b59f03e17ba None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:44.205789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076029ad-f20c-4aab-89ff-8b59f03e17ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:44.206438 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076029ad-f20c-4aab-89ff-8b59f03e17ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:44.208137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076029ad-f20c-4aab-89ff-8b59f03e17ba None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:44.229619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 381/762] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:44.235940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076029ad-f20c-4aab-89ff-8b59f03e17ba None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:44.237768 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:44.292726 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bb656-e479-4271-a9a5-b601487bf151 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:44.306504 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:44.345754 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3287365-0b82-41c2-9fb3-a8fd5f1a6cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:44.346439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3287365-0b82-41c2-9fb3-a8fd5f1a6cbc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:44.346714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3287365-0b82-41c2-9fb3-a8fd5f1a6cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:44.346984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3287365-0b82-41c2-9fb3-a8fd5f1a6cbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:44.365764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3287365-0b82-41c2-9fb3-a8fd5f1a6cbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:44.367990 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3287365-0b82-41c2-9fb3-a8fd5f1a6cbc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:44.379747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076029ad-f20c-4aab-89ff-8b59f03e17ba None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-d3287365-0b82-41c2-9fb3-a8fd5f1a6cbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:44.397635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:44.398648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:44.399538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:44.399538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:44.399538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:44.400417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 382/763] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:44.413276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3287365-0b82-41c2-9fb3-a8fd5f1a6cbc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:44.448759 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27066f4-7a11-4763-bc27-4623b05f21cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:44.450361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27066f4-7a11-4763-bc27-4623b05f21cb None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:44.450941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27066f4-7a11-4763-bc27-4623b05f21cb None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:44.451227 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27066f4-7a11-4763-bc27-4623b05f21cb None admin] PATH_INFO: `/v3/projects/32b06df0c5504592b012585f193386e7/users/1b30e80c421245b5a1ed5849b28ea90e/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:44.455562 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27066f4-7a11-4763-bc27-4623b05f21cb None admin] RBAC: Authorizing `identity:create_grant(project_id=32b06df0c5504592b012585f193386e7, user_id=1b30e80c421245b5a1ed5849b28ea90e, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:44.457028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27066f4-7a11-4763-bc27-4623b05f21cb None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:44.482220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 382/764] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:53:44 2022] PUT /identity/v3/projects/32b06df0c5504592b012585f193386e7/users/1b30e80c421245b5a1ed5849b28ea90e/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:53:44.566633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3287365-0b82-41c2-9fb3-a8fd5f1a6cbc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:44.579055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:44.600672 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27066f4-7a11-4763-bc27-4623b05f21cb None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:44.606327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9fd96-c38b-46bb-ae5e-4fceceb93300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:44.607142 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9fd96-c38b-46bb-ae5e-4fceceb93300 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:44.607499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9fd96-c38b-46bb-ae5e-4fceceb93300 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:44.607772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9fd96-c38b-46bb-ae5e-4fceceb93300 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:44.608435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9fd96-c38b-46bb-ae5e-4fceceb93300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:44.608992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9fd96-c38b-46bb-ae5e-4fceceb93300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:44.610503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9fd96-c38b-46bb-ae5e-4fceceb93300 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:44.625257 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 383/765] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:44.630922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9fd96-c38b-46bb-ae5e-4fceceb93300 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:44.758763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9fd96-c38b-46bb-ae5e-4fceceb93300 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:44.812977 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f17f3b-5309-4259-a5ff-2e8265c0e9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:44.813565 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27066f4-7a11-4763-bc27-4623b05f21cb None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:44.814007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f17f3b-5309-4259-a5ff-2e8265c0e9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:44.814007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f17f3b-5309-4259-a5ff-2e8265c0e9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:44.814197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f17f3b-5309-4259-a5ff-2e8265c0e9a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:44.822456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f17f3b-5309-4259-a5ff-2e8265c0e9a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:44.823820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f17f3b-5309-4259-a5ff-2e8265c0e9a4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:44.831991 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:44.837471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-69f17f3b-5309-4259-a5ff-2e8265c0e9a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:44.837471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:44.837471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:44.837471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:44.837471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:44.837471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:44.837471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:44.837471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:44.837471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:44.837471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:44.837471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:44.837471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:44.837471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:44.837471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:44.837471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:44.838393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:44.839308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:44.839308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:44.839308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:44.839308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:44.839308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:44.839308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:44.839308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:44.839308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:44.839308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 384/766] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:44.844706 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f17f3b-5309-4259-a5ff-2e8265c0e9a4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:44.867354 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35c46a-f37f-4b4f-96de-5dc7d2800ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:44.868001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35c46a-f37f-4b4f-96de-5dc7d2800ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:44.868278 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35c46a-f37f-4b4f-96de-5dc7d2800ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:44.868547 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35c46a-f37f-4b4f-96de-5dc7d2800ae6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:44.869145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35c46a-f37f-4b4f-96de-5dc7d2800ae6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:44.870554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35c46a-f37f-4b4f-96de-5dc7d2800ae6 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-2a35c46a-f37f-4b4f-96de-5dc7d2800ae6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:44.894720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:44.897056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:44.899064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:44.899064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:44.899064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:44.899064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:44.899064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:44.899064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:44.899064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:44.899064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:44.899064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:44.899064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:44.899064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:53:44.899064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:44.899064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:53:44.899064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:44.899064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 383/767] 10.222.0.62 () {56 vars in 1129 bytes} [Fri Sep 9 11:53:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:44.907030 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35c46a-f37f-4b4f-96de-5dc7d2800ae6 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:44.913620 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:45.047965 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f17f3b-5309-4259-a5ff-2e8265c0e9a4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:45.075031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35c46a-f37f-4b4f-96de-5dc7d2800ae6 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:45.127762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a380d-23b5-4b20-9a18-a45c49ac87dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:45.129086 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a380d-23b5-4b20-9a18-a45c49ac87dc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:45.129559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a380d-23b5-4b20-9a18-a45c49ac87dc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:45.130040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a380d-23b5-4b20-9a18-a45c49ac87dc None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:45.130963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a380d-23b5-4b20-9a18-a45c49ac87dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:45.131989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a380d-23b5-4b20-9a18-a45c49ac87dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:45.135972 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a380d-23b5-4b20-9a18-a45c49ac87dc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:45.156801 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 385/768] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:45.157235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081bcbc-3e78-4c1d-9a2a-2ba6cf9e449f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:45.157864 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081bcbc-3e78-4c1d-9a2a-2ba6cf9e449f None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:45.162412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081bcbc-3e78-4c1d-9a2a-2ba6cf9e449f None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:45.162412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081bcbc-3e78-4c1d-9a2a-2ba6cf9e449f None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:45.162412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081bcbc-3e78-4c1d-9a2a-2ba6cf9e449f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:45.162412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081bcbc-3e78-4c1d-9a2a-2ba6cf9e449f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:45.162412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081bcbc-3e78-4c1d-9a2a-2ba6cf9e449f None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:45.168961 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a380d-23b5-4b20-9a18-a45c49ac87dc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:45.170698 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 384/769] 10.222.0.62 () {56 vars in 1124 bytes} [Fri Sep 9 11:53:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:45.185750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081bcbc-3e78-4c1d-9a2a-2ba6cf9e449f None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:45.246130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:45.296729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:45.308960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a380d-23b5-4b20-9a18-a45c49ac87dc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:45.329987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081bcbc-3e78-4c1d-9a2a-2ba6cf9e449f None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:45.356200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0e603-339e-4548-93fa-4cb42ebe306c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:45.357503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0e603-339e-4548-93fa-4cb42ebe306c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:45.357921 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0e603-339e-4548-93fa-4cb42ebe306c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:45.358357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0e603-339e-4548-93fa-4cb42ebe306c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:45.359492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0e603-339e-4548-93fa-4cb42ebe306c None admin] RBAC: Adding query filter params (role.id=035f7d9f445d406eb2e83ca90fc4d5ca, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=01834ce862ae47cc85668b0a1c66f587) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:45.359936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0e603-339e-4548-93fa-4cb42ebe306c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:45.362379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0e603-339e-4548-93fa-4cb42ebe306c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:45.371256 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 386/770] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:45 2022] GET /identity/v3/role_assignments?user.id=01834ce862ae47cc85668b0a1c66f587&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=035f7d9f445d406eb2e83ca90fc4d5ca => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:45.373566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09903a-b6dc-47bb-89d6-1c5a048b43d0 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:45.374006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09903a-b6dc-47bb-89d6-1c5a048b43d0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:45.374434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09903a-b6dc-47bb-89d6-1c5a048b43d0 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:45.376845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 387/771] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:45.377618 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d2c52-6cd3-48b2-ac21-f5c0e24db01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:45.380863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d2c52-6cd3-48b2-ac21-f5c0e24db01d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:45.380863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d2c52-6cd3-48b2-ac21-f5c0e24db01d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:45.380863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d2c52-6cd3-48b2-ac21-f5c0e24db01d None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:45.380863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d2c52-6cd3-48b2-ac21-f5c0e24db01d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:45.380863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d2c52-6cd3-48b2-ac21-f5c0e24db01d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:45.386141 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d2c52-6cd3-48b2-ac21-f5c0e24db01d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:45.386322 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616db7d-6537-4b5a-83ac-c842c3825570 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:45.386322 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616db7d-6537-4b5a-83ac-c842c3825570 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:45.386322 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616db7d-6537-4b5a-83ac-c842c3825570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:45.391438 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 385/772] 10.222.0.62 () {56 vars in 1124 bytes} [Fri Sep 9 11:53:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:45.392758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621e9f0-69a3-4c55-9b7d-e1cbf7db6177 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:45.393040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621e9f0-69a3-4c55-9b7d-e1cbf7db6177 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:45.393345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621e9f0-69a3-4c55-9b7d-e1cbf7db6177 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:45.396452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 386/773] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:45.401533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621e9f0-69a3-4c55-9b7d-e1cbf7db6177 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:45.415111 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9616db7d-6537-4b5a-83ac-c842c3825570 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:45.416464 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9616db7d-6537-4b5a-83ac-c842c3825570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:45.503992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 388/774] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:53:45.505335 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd49d4-bb4e-474d-ade6-cb12cb052d53 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:45.505612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd49d4-bb4e-474d-ade6-cb12cb052d53 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:45.505877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd49d4-bb4e-474d-ade6-cb12cb052d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:45.528229 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-92cd49d4-bb4e-474d-ade6-cb12cb052d53 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:45.529417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-92cd49d4-bb4e-474d-ade6-cb12cb052d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:45.531478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621e9f0-69a3-4c55-9b7d-e1cbf7db6177 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:45.540863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:45.591269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d1823-cf23-4478-b4c5-b9079172c662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:45.592241 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d1823-cf23-4478-b4c5-b9079172c662 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:45.592619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d1823-cf23-4478-b4c5-b9079172c662 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:45.593390 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d1823-cf23-4478-b4c5-b9079172c662 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:45.594147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d1823-cf23-4478-b4c5-b9079172c662 None admin] RBAC: Adding query filter params (interface=public, service_id=18dbb3f281b642c6bd84fce69c3db014, region_id=RegionOne) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:45.594632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d1823-cf23-4478-b4c5-b9079172c662 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:45.596350 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d1823-cf23-4478-b4c5-b9079172c662 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:45.614619 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 389/775] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:53:45.620284 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 387/776] 10.222.0.62 () {56 vars in 1251 bytes} [Fri Sep 9 11:53:45 2022] GET /identity/v3/endpoints?service_id=18dbb3f281b642c6bd84fce69c3db014&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:45.621812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd49d4-bb4e-474d-ade6-cb12cb052d53 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:45.639270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:45.742220 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd49d4-bb4e-474d-ade6-cb12cb052d53 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:45.788098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44883e70-e784-415d-a142-3d9908eb9e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:45.788777 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44883e70-e784-415d-a142-3d9908eb9e4c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:45.789061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44883e70-e784-415d-a142-3d9908eb9e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:45.789334 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44883e70-e784-415d-a142-3d9908eb9e4c None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:45.789948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44883e70-e784-415d-a142-3d9908eb9e4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:45.791416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44883e70-e784-415d-a142-3d9908eb9e4c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:45.797107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 390/777] 10.222.0.62 () {56 vars in 1087 bytes} [Fri Sep 9 11:53:45 2022] GET /identity/v3/services => generated 772 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:45.821144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d1823-cf23-4478-b4c5-b9079172c662 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:45.955023 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d1823-cf23-4478-b4c5-b9079172c662 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:46.002565 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c428d-644e-4d9f-a0c6-8942ddbcc469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:46.003211 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c428d-644e-4d9f-a0c6-8942ddbcc469 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:46.003496 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c428d-644e-4d9f-a0c6-8942ddbcc469 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:46.003769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c428d-644e-4d9f-a0c6-8942ddbcc469 None admin] PATH_INFO: `/v3/services/network` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:46.004368 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c428d-644e-4d9f-a0c6-8942ddbcc469 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:46.005824 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c428d-644e-4d9f-a0c6-8942ddbcc469 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-ba3c428d-644e-4d9f-a0c6-8942ddbcc469 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:46.022790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:46.023833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:46.024678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:46.024678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:46.024678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:46.024678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:46.024678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:46.024678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:46.024678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:46.024678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:46.024678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:46.024678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:46.024678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:53:46.024678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:46.024678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:53:46.024678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:46.024678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 388/778] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:45 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:46.029885 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44883e70-e784-415d-a142-3d9908eb9e4c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:46.123931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:46.146705 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44883e70-e784-415d-a142-3d9908eb9e4c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:46.153461 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c428d-644e-4d9f-a0c6-8942ddbcc469 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:46.184038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bbede-19a2-4f60-bb5e-c2cacdbb3f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:46.185194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bbede-19a2-4f60-bb5e-c2cacdbb3f7c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:46.185666 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bbede-19a2-4f60-bb5e-c2cacdbb3f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:46.186152 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bbede-19a2-4f60-bb5e-c2cacdbb3f7c None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:46.186995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bbede-19a2-4f60-bb5e-c2cacdbb3f7c None admin] RBAC: Adding query filter params (name=network) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:46.187497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bbede-19a2-4f60-bb5e-c2cacdbb3f7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:46.189034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bbede-19a2-4f60-bb5e-c2cacdbb3f7c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:46.200971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 391/779] 10.222.0.62 () {56 vars in 1112 bytes} [Fri Sep 9 11:53:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:46.206422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bbede-19a2-4f60-bb5e-c2cacdbb3f7c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:46.284493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:46.320311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bbede-19a2-4f60-bb5e-c2cacdbb3f7c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:46.356104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12f3ca-01d8-491f-aa8b-2410ba8af6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:46.357067 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12f3ca-01d8-491f-aa8b-2410ba8af6db None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:46.357569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12f3ca-01d8-491f-aa8b-2410ba8af6db None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:46.358032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12f3ca-01d8-491f-aa8b-2410ba8af6db None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:46.359161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12f3ca-01d8-491f-aa8b-2410ba8af6db None admin] RBAC: Adding query filter params (type=network) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:46.359655 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12f3ca-01d8-491f-aa8b-2410ba8af6db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:46.361314 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12f3ca-01d8-491f-aa8b-2410ba8af6db None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:46.367572 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 392/780] 10.222.0.62 () {56 vars in 1112 bytes} [Fri Sep 9 11:53:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:46.373584 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12f3ca-01d8-491f-aa8b-2410ba8af6db None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:46.398558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:46.407472 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c428d-644e-4d9f-a0c6-8942ddbcc469 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:46.487925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12f3ca-01d8-491f-aa8b-2410ba8af6db None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:46.493380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8023f7c-c798-4080-aef5-e925bbb24744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:46.494025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8023f7c-c798-4080-aef5-e925bbb24744 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:46.494333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8023f7c-c798-4080-aef5-e925bbb24744 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:46.494605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8023f7c-c798-4080-aef5-e925bbb24744 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:46.519294 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8023f7c-c798-4080-aef5-e925bbb24744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-b8023f7c-c798-4080-aef5-e925bbb24744 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:46.525172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:46.526161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:46.527013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:46.527013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:46.527013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:46.527013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:46.527013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:46.527013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:46.527013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:46.527013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:46.527013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:46.527013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:46.527013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:46.527013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:46.530606 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac922b3-63b3-4a67-a916-7626dc0bc963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:46.531454 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac922b3-63b3-4a67-a916-7626dc0bc963 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:46.531924 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac922b3-63b3-4a67-a916-7626dc0bc963 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:46.532388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac922b3-63b3-4a67-a916-7626dc0bc963 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:46.533293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac922b3-63b3-4a67-a916-7626dc0bc963 None admin] RBAC: Adding query filter params (interface=public, service_id=51b91bdb399a4703a52fa30f62fa78d1, region_id=RegionOne) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:46.533791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac922b3-63b3-4a67-a916-7626dc0bc963 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:46.534611 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 389/781] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:53:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:46.535598 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac922b3-63b3-4a67-a916-7626dc0bc963 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:46.540143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8023f7c-c798-4080-aef5-e925bbb24744 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:46.545392 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 393/782] 10.222.0.62 () {56 vars in 1251 bytes} [Fri Sep 9 11:53:46 2022] GET /identity/v3/endpoints?service_id=51b91bdb399a4703a52fa30f62fa78d1&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:46.551810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac922b3-63b3-4a67-a916-7626dc0bc963 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:46.664910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:46.721249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac922b3-63b3-4a67-a916-7626dc0bc963 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:46.750221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:46.760010 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8023f7c-c798-4080-aef5-e925bbb24744 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:46.770828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf9aab-7a7e-478f-9774-4846e22937ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:46.771457 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf9aab-7a7e-478f-9774-4846e22937ce None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:46.771774 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf9aab-7a7e-478f-9774-4846e22937ce None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:46.772050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf9aab-7a7e-478f-9774-4846e22937ce None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:46.772780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf9aab-7a7e-478f-9774-4846e22937ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:46.774227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf9aab-7a7e-478f-9774-4846e22937ce None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:46.788701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 394/783] 10.222.0.62 () {56 vars in 1087 bytes} [Fri Sep 9 11:53:46 2022] GET /identity/v3/services => generated 772 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:46.804051 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8116a5-7a42-4498-84d9-ea7faaf1d089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:46.805007 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8116a5-7a42-4498-84d9-ea7faaf1d089 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:46.805500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8116a5-7a42-4498-84d9-ea7faaf1d089 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:46.805983 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8116a5-7a42-4498-84d9-ea7faaf1d089 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:46.806860 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8116a5-7a42-4498-84d9-ea7faaf1d089 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:46.807350 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8116a5-7a42-4498-84d9-ea7faaf1d089 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:46.814142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8116a5-7a42-4498-84d9-ea7faaf1d089 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:46.830076 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 390/784] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:46.835742 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf9aab-7a7e-478f-9774-4846e22937ce None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:46.972845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf9aab-7a7e-478f-9774-4846e22937ce None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:47.018659 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96aa21-2ceb-4989-96de-1d0c7e43b075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:47.019242 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96aa21-2ceb-4989-96de-1d0c7e43b075 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:47.019519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96aa21-2ceb-4989-96de-1d0c7e43b075 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:47.019792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96aa21-2ceb-4989-96de-1d0c7e43b075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:47.022633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:47.039229 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96aa21-2ceb-4989-96de-1d0c7e43b075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:47.040775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96aa21-2ceb-4989-96de-1d0c7e43b075 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-9b96aa21-2ceb-4989-96de-1d0c7e43b075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:47.063085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:47.064130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:47.065030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:47.065030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:47.065030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:47.065030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 395/785] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:47.070872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8116a5-7a42-4498-84d9-ea7faaf1d089 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:47.091509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e794b7e-64cc-4082-98f0-4d6afbbb9d2a None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:47.091851 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e794b7e-64cc-4082-98f0-4d6afbbb9d2a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:47.092056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e794b7e-64cc-4082-98f0-4d6afbbb9d2a None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:47.093682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 396/786] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:47.099325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947b1a3-45fd-4ef6-a585-be79b4047661 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:47.099607 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947b1a3-45fd-4ef6-a585-be79b4047661 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:47.100509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947b1a3-45fd-4ef6-a585-be79b4047661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:47.120462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-8947b1a3-45fd-4ef6-a585-be79b4047661 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:47.121659 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8947b1a3-45fd-4ef6-a585-be79b4047661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:47.202235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8116a5-7a42-4498-84d9-ea7faaf1d089 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:47.209348 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 397/787] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:53:47.238944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c45886-0a8f-4069-bd8a-03bcfed62f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:47.239557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c45886-0a8f-4069-bd8a-03bcfed62f09 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:47.239837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c45886-0a8f-4069-bd8a-03bcfed62f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:47.240109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c45886-0a8f-4069-bd8a-03bcfed62f09 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:47.241374 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c45886-0a8f-4069-bd8a-03bcfed62f09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:47.241681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c45886-0a8f-4069-bd8a-03bcfed62f09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:47.243117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c45886-0a8f-4069-bd8a-03bcfed62f09 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:47.256727 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 391/788] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:47.262319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947b1a3-45fd-4ef6-a585-be79b4047661 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:47.391312 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947b1a3-45fd-4ef6-a585-be79b4047661 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:47.416294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:47.439354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e1807-89c0-4c37-a3ad-ae6729f685d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:47.440068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e1807-89c0-4c37-a3ad-ae6729f685d2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:47.440353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e1807-89c0-4c37-a3ad-ae6729f685d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:47.440626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e1807-89c0-4c37-a3ad-ae6729f685d2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:47.450813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e1807-89c0-4c37-a3ad-ae6729f685d2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:47.452255 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e1807-89c0-4c37-a3ad-ae6729f685d2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-dd2e1807-89c0-4c37-a3ad-ae6729f685d2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:47.466072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:47.467112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:47.468066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:47.468066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:47.468066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:47.468066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:47.468066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:47.468066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:47.468066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:47.468066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:47.468066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:53:47.468066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:47.468066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 398/789] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:53:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:47.473275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c45886-0a8f-4069-bd8a-03bcfed62f09 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:47.494623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:47.528901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e1807-89c0-4c37-a3ad-ae6729f685d2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:47.613851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c45886-0a8f-4069-bd8a-03bcfed62f09 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:47.650629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1864c-169e-4a93-82dd-a761f30db795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:47.651159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1864c-169e-4a93-82dd-a761f30db795 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:47.651435 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1864c-169e-4a93-82dd-a761f30db795 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:47.651768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1864c-169e-4a93-82dd-a761f30db795 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:47.652511 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1864c-169e-4a93-82dd-a761f30db795 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:47.652815 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1864c-169e-4a93-82dd-a761f30db795 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:47.654274 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1864c-169e-4a93-82dd-a761f30db795 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:47.674339 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 392/790] 10.222.0.62 () {56 vars in 1135 bytes} [Fri Sep 9 11:53:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:47.680548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1864c-169e-4a93-82dd-a761f30db795 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:47.765318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:47.788307 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1864c-169e-4a93-82dd-a761f30db795 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:47.800200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e1807-89c0-4c37-a3ad-ae6729f685d2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:47.827037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69351f-ff2b-42b7-b05b-b0c7eff0b460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:47.828033 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69351f-ff2b-42b7-b05b-b0c7eff0b460 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:47.828517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69351f-ff2b-42b7-b05b-b0c7eff0b460 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:47.828995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69351f-ff2b-42b7-b05b-b0c7eff0b460 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:47.841174 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69351f-ff2b-42b7-b05b-b0c7eff0b460 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:47.843956 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69351f-ff2b-42b7-b05b-b0c7eff0b460 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-7f69351f-ff2b-42b7-b05b-b0c7eff0b460 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:47.861370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:47.862439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:47.863378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:47.863378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:47.863378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:47.866031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 393/791] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:47.872595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69351f-ff2b-42b7-b05b-b0c7eff0b460 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:47.950940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61e4d4-9f2f-4d67-b864-9b0725cbb451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:47.951518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61e4d4-9f2f-4d67-b864-9b0725cbb451 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:47.951804 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61e4d4-9f2f-4d67-b864-9b0725cbb451 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:47.952087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61e4d4-9f2f-4d67-b864-9b0725cbb451 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:47.988138 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61e4d4-9f2f-4d67-b864-9b0725cbb451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-1e61e4d4-9f2f-4d67-b864-9b0725cbb451 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:47.993576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:47.994559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:47.995439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:47.995439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:47.995439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:47.995439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:47.995439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:47.995439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:47.995439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:47.995439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:47.995439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:47.995439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:47.995439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:47.995439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:47.995439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 399/792] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:53:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 466 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:47.999180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69351f-ff2b-42b7-b05b-b0c7eff0b460 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:48.001897 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61e4d4-9f2f-4d67-b864-9b0725cbb451 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:48.052269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:48.065422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:48.066388 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e2651-2682-43d3-82fa-b35a6db6d405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:48.067010 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e2651-2682-43d3-82fa-b35a6db6d405 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:48.067288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e2651-2682-43d3-82fa-b35a6db6d405 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:48.067556 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e2651-2682-43d3-82fa-b35a6db6d405 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:48.068217 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e2651-2682-43d3-82fa-b35a6db6d405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:48.069070 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e2651-2682-43d3-82fa-b35a6db6d405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:48.070535 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e2651-2682-43d3-82fa-b35a6db6d405 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:48.228279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 394/793] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:48.233275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61e4d4-9f2f-4d67-b864-9b0725cbb451 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:48.236677 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e2651-2682-43d3-82fa-b35a6db6d405 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:48.286290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385dd63-d453-4dad-93c9-3f45f163f7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:48.286909 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385dd63-d453-4dad-93c9-3f45f163f7ed None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:48.287190 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385dd63-d453-4dad-93c9-3f45f163f7ed None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:48.288090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385dd63-d453-4dad-93c9-3f45f163f7ed None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:48.289291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385dd63-d453-4dad-93c9-3f45f163f7ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:48.289861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385dd63-d453-4dad-93c9-3f45f163f7ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:48.291504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385dd63-d453-4dad-93c9-3f45f163f7ed None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:48.301363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 400/794] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:48.302810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012c019-f122-4819-9fc9-a800352f89ff None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:48.303722 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012c019-f122-4819-9fc9-a800352f89ff None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:48.304004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012c019-f122-4819-9fc9-a800352f89ff None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:48.309391 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 401/795] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:48.309391 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012c019-f122-4819-9fc9-a800352f89ff None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:48.363170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e2651-2682-43d3-82fa-b35a6db6d405 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:48.400984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17618c-c154-4766-a6b5-a90783780b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:48.401987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17618c-c154-4766-a6b5-a90783780b8d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:48.402505 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17618c-c154-4766-a6b5-a90783780b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:48.402954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17618c-c154-4766-a6b5-a90783780b8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:48.411586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17618c-c154-4766-a6b5-a90783780b8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:48.413634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17618c-c154-4766-a6b5-a90783780b8d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:48.425554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-dd17618c-c154-4766-a6b5-a90783780b8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:48.425554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:48.425554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:48.425554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:48.425554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:48.425554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:48.425554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:48.425554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:48.425554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:48.425554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:48.425554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:48.425554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:48.425554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:48.425554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:48.425554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:48.426558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:48.427483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:48.427483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:48.427483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:48.427483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:48.427483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:48.427483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:48.427483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:48.427483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:48.431695 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 395/796] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:48.433314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd6e62-e6f8-449c-8a97-5b6b0901d0db None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:48.433451 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd6e62-e6f8-449c-8a97-5b6b0901d0db None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:48.433616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd6e62-e6f8-449c-8a97-5b6b0901d0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:48.436919 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:48.439937 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012c019-f122-4819-9fc9-a800352f89ff None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:48.464141 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-b8fd6e62-e6f8-449c-8a97-5b6b0901d0db None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:48.466442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b8fd6e62-e6f8-449c-8a97-5b6b0901d0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:48.481989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f47043-392e-46a9-879d-b97e60ffda7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:48.483004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f47043-392e-46a9-879d-b97e60ffda7a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:48.483459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f47043-392e-46a9-879d-b97e60ffda7a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:48.484211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f47043-392e-46a9-879d-b97e60ffda7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:48.495832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f47043-392e-46a9-879d-b97e60ffda7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:48.497630 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f47043-392e-46a9-879d-b97e60ffda7a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-d1f47043-392e-46a9-879d-b97e60ffda7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:48.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:48.515016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:48.515906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:48.515906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:48.515906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:48.517982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 402/797] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:48.519077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f47043-392e-46a9-879d-b97e60ffda7a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:48.522386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:48.562204 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 396/798] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:53:48.563487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd6e62-e6f8-449c-8a97-5b6b0901d0db None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:48.642925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f47043-392e-46a9-879d-b97e60ffda7a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:48.688740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d404c48-ab0d-4811-82be-145502c88b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:48.689351 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d404c48-ab0d-4811-82be-145502c88b2e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:48.689624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d404c48-ab0d-4811-82be-145502c88b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:48.689888 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d404c48-ab0d-4811-82be-145502c88b2e None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:48.690245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd6e62-e6f8-449c-8a97-5b6b0901d0db None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:48.690601 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d404c48-ab0d-4811-82be-145502c88b2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:48.690897 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d404c48-ab0d-4811-82be-145502c88b2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:48.692560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d404c48-ab0d-4811-82be-145502c88b2e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:48.709806 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 403/799] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:48.711087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e301c-4fba-47f7-99ef-2827b609f78f None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:48.711368 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e301c-4fba-47f7-99ef-2827b609f78f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:48.711638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e301c-4fba-47f7-99ef-2827b609f78f None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:48.714240 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 404/800] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:48.715209 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e301c-4fba-47f7-99ef-2827b609f78f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:48.733033 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16203fa-be67-4c26-8592-2edfb9bb7819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:48.734345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16203fa-be67-4c26-8592-2edfb9bb7819 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:48.734598 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:48.735413 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16203fa-be67-4c26-8592-2edfb9bb7819 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:48.736166 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16203fa-be67-4c26-8592-2edfb9bb7819 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:48.737340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16203fa-be67-4c26-8592-2edfb9bb7819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:48.738313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16203fa-be67-4c26-8592-2edfb9bb7819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:48.740148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16203fa-be67-4c26-8592-2edfb9bb7819 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:48.755060 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 397/801] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:48.756777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed830aee-1180-46a0-aa64-f3c4234ab81e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:48.757259 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed830aee-1180-46a0-aa64-f3c4234ab81e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:48.757747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed830aee-1180-46a0-aa64-f3c4234ab81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:48.777519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-ed830aee-1180-46a0-aa64-f3c4234ab81e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:48.779094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ed830aee-1180-46a0-aa64-f3c4234ab81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:48.816015 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:48.859175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 398/802] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:53:48.860241 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed830aee-1180-46a0-aa64-f3c4234ab81e None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:48.867943 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e301c-4fba-47f7-99ef-2827b609f78f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:48.904856 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045402df-e50f-400d-815d-3735d1c130d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:48.905957 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045402df-e50f-400d-815d-3735d1c130d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:48.906590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045402df-e50f-400d-815d-3735d1c130d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:48.907069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045402df-e50f-400d-815d-3735d1c130d4 None admin] PATH_INFO: `/v3/projects/32b06df0c5504592b012585f193386e7/users/1b2de52919e54bf9bcc352909c9f905f/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:48.909145 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045402df-e50f-400d-815d-3735d1c130d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=32b06df0c5504592b012585f193386e7, user_id=1b2de52919e54bf9bcc352909c9f905f, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:48.910747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045402df-e50f-400d-815d-3735d1c130d4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:48.932635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 405/803] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:53:48 2022] PUT /identity/v3/projects/32b06df0c5504592b012585f193386e7/users/1b2de52919e54bf9bcc352909c9f905f/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:48.932635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045402df-e50f-400d-815d-3735d1c130d4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:49.026546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed830aee-1180-46a0-aa64-f3c4234ab81e None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:49.078413 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:49.099160 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a4e8a-ffde-4348-9692-6db1de5135b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:49.099849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a4e8a-ffde-4348-9692-6db1de5135b3 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:49.100126 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a4e8a-ffde-4348-9692-6db1de5135b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:49.100391 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a4e8a-ffde-4348-9692-6db1de5135b3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:49.115042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a4e8a-ffde-4348-9692-6db1de5135b3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:49.116864 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a4e8a-ffde-4348-9692-6db1de5135b3 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:49.139597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045402df-e50f-400d-815d-3735d1c130d4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-cb2a4e8a-ffde-4348-9692-6db1de5135b3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:49.141905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:49.142883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:49.143885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:49.143885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:49.143885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:49.143885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:49.143885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:49.143885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:49.143885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:49.143885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:49.143885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:53:49.143885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:49.143885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 399/804] 10.222.0.62 () {56 vars in 1093 bytes} [Fri Sep 9 11:53:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:49.149318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a4e8a-ffde-4348-9692-6db1de5135b3 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:49.184986 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:49.196265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd8905-bb01-4419-9c95-906b743fd9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:49.196931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd8905-bb01-4419-9c95-906b743fd9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:49.197208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd8905-bb01-4419-9c95-906b743fd9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:49.197476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd8905-bb01-4419-9c95-906b743fd9f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:49.218919 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd8905-bb01-4419-9c95-906b743fd9f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-50dd8905-bb01-4419-9c95-906b743fd9f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:49.228521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:49.229470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:49.230345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:49.230345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:49.230345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:49.230345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:49.230345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:49.230345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:49.230345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:49.230345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:49.230345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:49.230345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:49.230345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:49.230345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:49.230345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 406/805] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:49.234946 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd8905-bb01-4419-9c95-906b743fd9f4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:49.290328 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a4e8a-ffde-4348-9692-6db1de5135b3 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:49.337520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefa4c1-b00b-486f-90c6-eacc0bffa55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:49.338985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefa4c1-b00b-486f-90c6-eacc0bffa55c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:49.339752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefa4c1-b00b-486f-90c6-eacc0bffa55c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:49.340221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefa4c1-b00b-486f-90c6-eacc0bffa55c None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:49.341322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefa4c1-b00b-486f-90c6-eacc0bffa55c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:49.342157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefa4c1-b00b-486f-90c6-eacc0bffa55c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:49.344248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefa4c1-b00b-486f-90c6-eacc0bffa55c None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:49.344682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd8905-bb01-4419-9c95-906b743fd9f4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:49.373593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 400/806] 10.222.0.62 () {56 vars in 1133 bytes} [Fri Sep 9 11:53:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:49.373593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefa4c1-b00b-486f-90c6-eacc0bffa55c None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:49.384370 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:49.394640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e3fb-0cb5-4cbf-a1c0-eb8473573b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:49.395929 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e3fb-0cb5-4cbf-a1c0-eb8473573b48 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:49.396913 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e3fb-0cb5-4cbf-a1c0-eb8473573b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:49.397385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e3fb-0cb5-4cbf-a1c0-eb8473573b48 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:49.398260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8e3fb-0cb5-4cbf-a1c0-eb8473573b48 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:49.400365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8e3fb-0cb5-4cbf-a1c0-eb8473573b48 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-32b8e3fb-0cb5-4cbf-a1c0-eb8473573b48 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:49.414079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:49.415135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:49.415960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:49.415960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:49.415960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:49.415960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:49.415960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:49.415960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:49.415960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:49.415960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:49.415960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:49.415960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:49.415960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:53:49.415960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:49.415960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 11:53:49.415960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:49.418169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 407/807] 10.222.0.62 () {56 vars in 1129 bytes} [Fri Sep 9 11:53:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:49.418966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e3fb-0cb5-4cbf-a1c0-eb8473573b48 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:49.470848 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:49.490795 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefa4c1-b00b-486f-90c6-eacc0bffa55c None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:49.526656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e3fb-0cb5-4cbf-a1c0-eb8473573b48 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:49.530464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194187f-da45-4d0d-8e3b-ae9ae3afb382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:49.531069 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194187f-da45-4d0d-8e3b-ae9ae3afb382 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:49.531345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194187f-da45-4d0d-8e3b-ae9ae3afb382 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:49.531616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194187f-da45-4d0d-8e3b-ae9ae3afb382 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:49.532284 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194187f-da45-4d0d-8e3b-ae9ae3afb382 None admin] RBAC: Adding query filter params (name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:49.532584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194187f-da45-4d0d-8e3b-ae9ae3afb382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:49.533945 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194187f-da45-4d0d-8e3b-ae9ae3afb382 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:49.550360 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 401/808] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:49.551942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7f3be-a2b9-46c9-82ce-65c8518035a5 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:49.552225 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7f3be-a2b9-46c9-82ce-65c8518035a5 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:49.552502 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7f3be-a2b9-46c9-82ce-65c8518035a5 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:49.554667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 402/809] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:49.555327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7f3be-a2b9-46c9-82ce-65c8518035a5 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:49.570183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fad86f-5081-4985-8fa5-7c7d56f09de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:49.570977 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fad86f-5081-4985-8fa5-7c7d56f09de9 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:49.571254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fad86f-5081-4985-8fa5-7c7d56f09de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:49.571931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fad86f-5081-4985-8fa5-7c7d56f09de9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:49.636221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:49.645879 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fad86f-5081-4985-8fa5-7c7d56f09de9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:49.647408 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fad86f-5081-4985-8fa5-7c7d56f09de9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:49.676631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7f3be-a2b9-46c9-82ce-65c8518035a5 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:49.711279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6d33c-83e7-4313-bcca-452556569038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:49.711985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6d33c-83e7-4313-bcca-452556569038 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:49.712266 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6d33c-83e7-4313-bcca-452556569038 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:49.712537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6d33c-83e7-4313-bcca-452556569038 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:49.713168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6d33c-83e7-4313-bcca-452556569038 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:49.713468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6d33c-83e7-4313-bcca-452556569038 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:49.714890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6d33c-83e7-4313-bcca-452556569038 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:49.719650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 403/810] 10.222.0.62 () {56 vars in 1124 bytes} [Fri Sep 9 11:53:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:49.720452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6d33c-83e7-4313-bcca-452556569038 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:49.735497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:49.772124 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 408/811] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:53:49.773645 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5b853-bfdb-4d68-ac18-cb842c881dae None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:49.774059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5b853-bfdb-4d68-ac18-cb842c881dae None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:49.774502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5b853-bfdb-4d68-ac18-cb842c881dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:49.794069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1da5b853-bfdb-4d68-ac18-cb842c881dae None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:49.795284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1da5b853-bfdb-4d68-ac18-cb842c881dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:49.839329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6d33c-83e7-4313-bcca-452556569038 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:49.876186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06973081-fc56-4736-9475-952bd3b880d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:49.877007 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06973081-fc56-4736-9475-952bd3b880d7 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:49.877296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06973081-fc56-4736-9475-952bd3b880d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:49.877575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06973081-fc56-4736-9475-952bd3b880d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:49.878460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 409/812] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:53:49.879501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5b853-bfdb-4d68-ac18-cb842c881dae None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:49.891440 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06973081-fc56-4736-9475-952bd3b880d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:49.893319 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06973081-fc56-4736-9475-952bd3b880d7 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-06973081-fc56-4736-9475-952bd3b880d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:49.909487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:49.910551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:49.911437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:49.911437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:49.911437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:49.911437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 404/813] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:49.911661 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06973081-fc56-4736-9475-952bd3b880d7 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:49.917936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:49.980093 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:50.019514 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5b853-bfdb-4d68-ac18-cb842c881dae None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:50.021319 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06973081-fc56-4736-9475-952bd3b880d7 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:50.057601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ffaea-28ed-4ad6-8352-79b24cf4a43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:50.058443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ffaea-28ed-4ad6-8352-79b24cf4a43a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:50.058814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ffaea-28ed-4ad6-8352-79b24cf4a43a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:50.059226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ffaea-28ed-4ad6-8352-79b24cf4a43a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:50.065227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a77e2c-7958-437e-ac5f-6e61d85a0eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:50.065831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a77e2c-7958-437e-ac5f-6e61d85a0eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:50.066102 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a77e2c-7958-437e-ac5f-6e61d85a0eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:50.066393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a77e2c-7958-437e-ac5f-6e61d85a0eb1 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:50.067015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a77e2c-7958-437e-ac5f-6e61d85a0eb1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:50.067316 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a77e2c-7958-437e-ac5f-6e61d85a0eb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:50.068820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a77e2c-7958-437e-ac5f-6e61d85a0eb1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:50.074050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 410/814] 10.222.0.62 () {56 vars in 1124 bytes} [Fri Sep 9 11:53:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:50.074879 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a77e2c-7958-437e-ac5f-6e61d85a0eb1 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:50.080252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ffaea-28ed-4ad6-8352-79b24cf4a43a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:50.081942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ffaea-28ed-4ad6-8352-79b24cf4a43a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-020ffaea-28ed-4ad6-8352-79b24cf4a43a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:50.098580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:50.099548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:50.100586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:50.100586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:50.100586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:50.107161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 405/815] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:50.108379 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ffaea-28ed-4ad6-8352-79b24cf4a43a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:50.185235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:50.204983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a77e2c-7958-437e-ac5f-6e61d85a0eb1 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:50.221173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ffaea-28ed-4ad6-8352-79b24cf4a43a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:50.237676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:50.248869 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8499a-02be-4f57-b4fc-306c81255bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:50.249482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8499a-02be-4f57-b4fc-306c81255bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:50.249763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8499a-02be-4f57-b4fc-306c81255bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:50.250032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8499a-02be-4f57-b4fc-306c81255bb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:50.250700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8499a-02be-4f57-b4fc-306c81255bb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:50.251000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8499a-02be-4f57-b4fc-306c81255bb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:50.252350 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8499a-02be-4f57-b4fc-306c81255bb4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:50.260322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef861e-ab6a-4cf5-853a-981d485d78da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:50.260676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef861e-ab6a-4cf5-853a-981d485d78da None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:50.260947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef861e-ab6a-4cf5-853a-981d485d78da None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:50.261215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef861e-ab6a-4cf5-853a-981d485d78da None admin] PATH_INFO: `/v3/endpoints` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:50.261880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef861e-ab6a-4cf5-853a-981d485d78da None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:50.263291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef861e-ab6a-4cf5-853a-981d485d78da None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:50.265820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 411/816] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:50.267201 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8499a-02be-4f57-b4fc-306c81255bb4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:50.286386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 406/817] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 11:53:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:50.287756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef861e-ab6a-4cf5-853a-981d485d78da None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:50.410891 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8499a-02be-4f57-b4fc-306c81255bb4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:50.418726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef861e-ab6a-4cf5-853a-981d485d78da None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:50.453896 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2c4bb-f1a6-4b0f-9d9b-d9ec2eb9ff61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:50.454750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2c4bb-f1a6-4b0f-9d9b-d9ec2eb9ff61 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:50.455111 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2c4bb-f1a6-4b0f-9d9b-d9ec2eb9ff61 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:50.455485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2c4bb-f1a6-4b0f-9d9b-d9ec2eb9ff61 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:50.456243 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2c4bb-f1a6-4b0f-9d9b-d9ec2eb9ff61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:50.456918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2c4bb-f1a6-4b0f-9d9b-d9ec2eb9ff61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:50.459712 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2c4bb-f1a6-4b0f-9d9b-d9ec2eb9ff61 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:50.464637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d663b7-3c32-4108-8c0a-3968838dd752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:50.465602 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d663b7-3c32-4108-8c0a-3968838dd752 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:50.466065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d663b7-3c32-4108-8c0a-3968838dd752 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:50.466532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d663b7-3c32-4108-8c0a-3968838dd752 None admin] PATH_INFO: `/v3/services/network` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:50.467269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d663b7-3c32-4108-8c0a-3968838dd752 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:50.468836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d663b7-3c32-4108-8c0a-3968838dd752 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:50.475690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 412/818] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:50.476691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2c4bb-f1a6-4b0f-9d9b-d9ec2eb9ff61 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-31d663b7-3c32-4108-8c0a-3968838dd752 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:50.482309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:50.483788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:50.484849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:50.484849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:50.484849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:50.484849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:50.484849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:50.484849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:50.484849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:50.484849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:50.484849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:50.484849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:50.484849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:53:50.484849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:50.484849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 11:53:50.484849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:50.487450 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 407/819] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:50 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:50.488472 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d663b7-3c32-4108-8c0a-3968838dd752 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:50.492574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:50.531204 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:50.607318 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2c4bb-f1a6-4b0f-9d9b-d9ec2eb9ff61 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:50.613399 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d663b7-3c32-4108-8c0a-3968838dd752 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:50.650656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2115f21-0921-46e1-8464-a23d12390bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:50.651790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2115f21-0921-46e1-8464-a23d12390bad None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:50.652249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2115f21-0921-46e1-8464-a23d12390bad None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:50.652356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f62d17-a8aa-4010-9cb9-43d67c2f1fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:50.652807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2115f21-0921-46e1-8464-a23d12390bad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:50.653069 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f62d17-a8aa-4010-9cb9-43d67c2f1fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:50.653423 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f62d17-a8aa-4010-9cb9-43d67c2f1fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:50.653764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f62d17-a8aa-4010-9cb9-43d67c2f1fb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:50.661695 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f62d17-a8aa-4010-9cb9-43d67c2f1fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:50.662102 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2115f21-0921-46e1-8464-a23d12390bad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:50.663341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f62d17-a8aa-4010-9cb9-43d67c2f1fb1 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:50.664277 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2115f21-0921-46e1-8464-a23d12390bad None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:50.677693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-78f62d17-a8aa-4010-9cb9-43d67c2f1fb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:50.677693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:50.677693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:50.677693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:50.677693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:50.677693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:50.677693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:50.677693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:50.677693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:50.677693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:50.677693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:50.677693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:50.677693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:50.677693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:50.677693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:50.679182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:50.680073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:50.680073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:50.680073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:50.680073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:50.680073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:50.680073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:50.680073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:50.680073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-d2115f21-0921-46e1-8464-a23d12390bad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:50.680502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:50.681360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:50.683014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:50.683014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:50.683014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:50.683014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:50.683014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:50.683014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:50.683014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:50.683014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:50.683014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:53:50.683014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:50.685434 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 408/821] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:50.685763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 413/821] 10.222.0.62 () {56 vars in 1117 bytes} [Fri Sep 9 11:53:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:50.686657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f62d17-a8aa-4010-9cb9-43d67c2f1fb1 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:50.692447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2115f21-0921-46e1-8464-a23d12390bad None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:50.771465 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:50.790909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:50.801829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f62d17-a8aa-4010-9cb9-43d67c2f1fb1 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:50.816825 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2115f21-0921-46e1-8464-a23d12390bad None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:50.837638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a304de3-d885-4647-9186-31cca30f5974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:50.838498 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a304de3-d885-4647-9186-31cca30f5974 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:50.838782 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a304de3-d885-4647-9186-31cca30f5974 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:50.839057 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a304de3-d885-4647-9186-31cca30f5974 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:50.839699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a304de3-d885-4647-9186-31cca30f5974 None admin] RBAC: Adding query filter params (name=network) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:50.840004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a304de3-d885-4647-9186-31cca30f5974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:50.841492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a304de3-d885-4647-9186-31cca30f5974 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:50.847462 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 409/822] 10.222.0.62 () {56 vars in 1112 bytes} [Fri Sep 9 11:53:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:50.848543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a304de3-d885-4647-9186-31cca30f5974 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:50.853879 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbed4c7-28e7-4e52-b5e4-36a53ed85ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:50.854529 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbed4c7-28e7-4e52-b5e4-36a53ed85ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:50.854799 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbed4c7-28e7-4e52-b5e4-36a53ed85ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:50.855067 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbed4c7-28e7-4e52-b5e4-36a53ed85ca3 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:50.855762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbed4c7-28e7-4e52-b5e4-36a53ed85ca3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:50.856062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbed4c7-28e7-4e52-b5e4-36a53ed85ca3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:50.857476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbed4c7-28e7-4e52-b5e4-36a53ed85ca3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:50.871508 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 414/823] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:50.872371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbed4c7-28e7-4e52-b5e4-36a53ed85ca3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:50.962055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a304de3-d885-4647-9186-31cca30f5974 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:50.985656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbed4c7-28e7-4e52-b5e4-36a53ed85ca3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:50.998001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f163610-abd6-4b87-b447-1c46ee898c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:50.998639 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f163610-abd6-4b87-b447-1c46ee898c5b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:50.998917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f163610-abd6-4b87-b447-1c46ee898c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:50.999188 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f163610-abd6-4b87-b447-1c46ee898c5b None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:50.999919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f163610-abd6-4b87-b447-1c46ee898c5b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:51.000285 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f163610-abd6-4b87-b447-1c46ee898c5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:51.001771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f163610-abd6-4b87-b447-1c46ee898c5b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:51.018200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:51.025359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 410/824] 10.222.0.62 () {56 vars in 1149 bytes} [Fri Sep 9 11:53:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:51.026325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f163610-abd6-4b87-b447-1c46ee898c5b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:51.029357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bb607-ac60-46ed-a4d9-ea1f1811a4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:51.030269 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bb607-ac60-46ed-a4d9-ea1f1811a4cf None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:51.030686 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bb607-ac60-46ed-a4d9-ea1f1811a4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:51.031111 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bb607-ac60-46ed-a4d9-ea1f1811a4cf None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:51.031897 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bb607-ac60-46ed-a4d9-ea1f1811a4cf None admin] RBAC: Adding query filter params (type=network) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:51.032339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bb607-ac60-46ed-a4d9-ea1f1811a4cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:51.033825 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bb607-ac60-46ed-a4d9-ea1f1811a4cf None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:51.039944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 415/825] 10.222.0.62 () {56 vars in 1112 bytes} [Fri Sep 9 11:53:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:51.041042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bb607-ac60-46ed-a4d9-ea1f1811a4cf None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:51.042111 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:51.143781 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f163610-abd6-4b87-b447-1c46ee898c5b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:51.179780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bb607-ac60-46ed-a4d9-ea1f1811a4cf None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:51.184601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdd6c2-2fa7-4253-9cbe-33c7e6825cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:51.185211 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdd6c2-2fa7-4253-9cbe-33c7e6825cfa None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:51.185487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdd6c2-2fa7-4253-9cbe-33c7e6825cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:51.185757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdd6c2-2fa7-4253-9cbe-33c7e6825cfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:51.186590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdd6c2-2fa7-4253-9cbe-33c7e6825cfa None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=1b30e80c421245b5a1ed5849b28ea90e) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:51.186905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdd6c2-2fa7-4253-9cbe-33c7e6825cfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:51.188336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdd6c2-2fa7-4253-9cbe-33c7e6825cfa None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:51.206294 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 411/826] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:51 2022] GET /identity/v3/role_assignments?user.id=1b30e80c421245b5a1ed5849b28ea90e&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:51.207370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdd6c2-2fa7-4253-9cbe-33c7e6825cfa None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:51.231090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05204224-c27e-4b31-b8ee-e250b5e23fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:51.232215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05204224-c27e-4b31-b8ee-e250b5e23fbc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:51.234506 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05204224-c27e-4b31-b8ee-e250b5e23fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:51.234995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05204224-c27e-4b31-b8ee-e250b5e23fbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:51.262862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05204224-c27e-4b31-b8ee-e250b5e23fbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:51.264318 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05204224-c27e-4b31-b8ee-e250b5e23fbc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-05204224-c27e-4b31-b8ee-e250b5e23fbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:51.284563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:51.285700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:51.286627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:51.286627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:51.286627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:51.286627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 416/827] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:51.287425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed2277-4679-4bac-be12-aa11ade60518 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:51.287653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed2277-4679-4bac-be12-aa11ade60518 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:51.287921 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed2277-4679-4bac-be12-aa11ade60518 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:51.293383 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 417/828] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:51.295678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f146e83-d4c1-4f73-955a-fdf0cfcb45ad None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:51.296208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f146e83-d4c1-4f73-955a-fdf0cfcb45ad None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:51.296689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f146e83-d4c1-4f73-955a-fdf0cfcb45ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:51.320874 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7f146e83-d4c1-4f73-955a-fdf0cfcb45ad None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:51.322660 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7f146e83-d4c1-4f73-955a-fdf0cfcb45ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:51.337560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:51.365659 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdd6c2-2fa7-4253-9cbe-33c7e6825cfa None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:51.384912 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:51.405890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193878ba-26d5-4f99-8ee4-4bf6883d44f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:51.406445 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193878ba-26d5-4f99-8ee4-4bf6883d44f6 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:51.406725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193878ba-26d5-4f99-8ee4-4bf6883d44f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:51.406997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193878ba-26d5-4f99-8ee4-4bf6883d44f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:51.407665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193878ba-26d5-4f99-8ee4-4bf6883d44f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:51.409026 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193878ba-26d5-4f99-8ee4-4bf6883d44f6 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:51.424420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 418/829] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:53:51.424420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f146e83-d4c1-4f73-955a-fdf0cfcb45ad None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:51.447956 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 412/830] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 11:53:51 2022] POST /identity/v3/endpoints => generated 337 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:53:51.552295 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f146e83-d4c1-4f73-955a-fdf0cfcb45ad None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:51.600930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21301710-6652-4970-b4b2-3b10b0576612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:51.601667 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21301710-6652-4970-b4b2-3b10b0576612 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:51.602009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21301710-6652-4970-b4b2-3b10b0576612 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:51.602469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21301710-6652-4970-b4b2-3b10b0576612 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:51.603198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21301710-6652-4970-b4b2-3b10b0576612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:51.603569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21301710-6652-4970-b4b2-3b10b0576612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:51.605018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21301710-6652-4970-b4b2-3b10b0576612 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:51.621424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 419/831] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:51.625097 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193878ba-26d5-4f99-8ee4-4bf6883d44f6 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:51.728603 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21301710-6652-4970-b4b2-3b10b0576612 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:51.746301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193878ba-26d5-4f99-8ee4-4bf6883d44f6 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:51.764346 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:51.782294 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:51.790554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae38ea6-1385-48c8-a3b2-54b6748296f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:51.791774 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae38ea6-1385-48c8-a3b2-54b6748296f9 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:51.792410 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae38ea6-1385-48c8-a3b2-54b6748296f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:51.793053 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae38ea6-1385-48c8-a3b2-54b6748296f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:51.802180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae38ea6-1385-48c8-a3b2-54b6748296f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:51.803951 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae38ea6-1385-48c8-a3b2-54b6748296f9 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:51.816716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-1ae38ea6-1385-48c8-a3b2-54b6748296f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:51.816716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:51.816716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:51.816716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:51.816716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:51.816716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:51.816716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:51.816716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:51.816716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:51.816716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:51.816716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:51.816716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:51.816716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:51.816716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:51.816716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:51.817845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:51.819667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:51.819667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:51.819667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:51.819667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:51.819667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:51.819667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:51.819667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:51.819667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:51.823756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 413/832] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:51.825929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae38ea6-1385-48c8-a3b2-54b6748296f9 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:51.860773 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21301710-6652-4970-b4b2-3b10b0576612 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:51.904748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fa859-0f2a-4641-9319-61be9e90bd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:51.905769 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fa859-0f2a-4641-9319-61be9e90bd1f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:51.906308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fa859-0f2a-4641-9319-61be9e90bd1f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:51.906794 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fa859-0f2a-4641-9319-61be9e90bd1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:51.918161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fa859-0f2a-4641-9319-61be9e90bd1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-274fa859-0f2a-4641-9319-61be9e90bd1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:51.923813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:51.924766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:51.925650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:51.925650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:51.925650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:51.925650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:51.925650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:51.925650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:51.925650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:51.925650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:51.925650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:51.925650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:51.925650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:53:51.925650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:51.933113 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 420/833] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:53:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:51.933591 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fa859-0f2a-4641-9319-61be9e90bd1f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:51.967931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae38ea6-1385-48c8-a3b2-54b6748296f9 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:52.019224 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248a5e3-3b4d-4d5d-8801-4c5eae04b433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:52.020126 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248a5e3-3b4d-4d5d-8801-4c5eae04b433 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.020613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248a5e3-3b4d-4d5d-8801-4c5eae04b433 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.021034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248a5e3-3b4d-4d5d-8801-4c5eae04b433 None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.022005 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248a5e3-3b4d-4d5d-8801-4c5eae04b433 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:52.022478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248a5e3-3b4d-4d5d-8801-4c5eae04b433 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:52.024068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248a5e3-3b4d-4d5d-8801-4c5eae04b433 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:52.042281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 414/834] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:52.048263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:52.050816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248a5e3-3b4d-4d5d-8801-4c5eae04b433 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:52.062402 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fa859-0f2a-4641-9319-61be9e90bd1f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:52.110443 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86b4b3-2ef4-45c4-9e70-51613c65f98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:52.111071 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86b4b3-2ef4-45c4-9e70-51613c65f98d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.111356 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86b4b3-2ef4-45c4-9e70-51613c65f98d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.111626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86b4b3-2ef4-45c4-9e70-51613c65f98d None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.112299 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f86b4b3-2ef4-45c4-9e70-51613c65f98d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:52.112595 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f86b4b3-2ef4-45c4-9e70-51613c65f98d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:52.113946 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f86b4b3-2ef4-45c4-9e70-51613c65f98d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:52.118301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:52.128267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 421/835] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:53:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:52.134711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86b4b3-2ef4-45c4-9e70-51613c65f98d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:52.187235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248a5e3-3b4d-4d5d-8801-4c5eae04b433 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:52.237436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b86c6f-712f-4fa5-9c8b-97ffd18c6d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:52.238656 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b86c6f-712f-4fa5-9c8b-97ffd18c6d1a None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.239398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b86c6f-712f-4fa5-9c8b-97ffd18c6d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.239911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b86c6f-712f-4fa5-9c8b-97ffd18c6d1a None admin] PATH_INFO: `/v3/projects/32b06df0c5504592b012585f193386e7/users/01834ce862ae47cc85668b0a1c66f587/roles/035f7d9f445d406eb2e83ca90fc4d5ca` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.257458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86b4b3-2ef4-45c4-9e70-51613c65f98d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:52.261578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b86c6f-712f-4fa5-9c8b-97ffd18c6d1a None admin] RBAC: Authorizing `identity:create_grant(project_id=32b06df0c5504592b012585f193386e7, user_id=01834ce862ae47cc85668b0a1c66f587, role_id=035f7d9f445d406eb2e83ca90fc4d5ca)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:52.263262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b86c6f-712f-4fa5-9c8b-97ffd18c6d1a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:52.282266 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 415/836] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:53:52 2022] PUT /identity/v3/projects/32b06df0c5504592b012585f193386e7/users/01834ce862ae47cc85668b0a1c66f587/roles/035f7d9f445d406eb2e83ca90fc4d5ca => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:53:52.309271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fae62-de2b-436c-949c-f978ada60f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:52.309937 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fae62-de2b-436c-949c-f978ada60f17 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.310237 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fae62-de2b-436c-949c-f978ada60f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.310511 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fae62-de2b-436c-949c-f978ada60f17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.324224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fae62-de2b-436c-949c-f978ada60f17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:52.325608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fae62-de2b-436c-949c-f978ada60f17 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-0a9fae62-de2b-436c-949c-f978ada60f17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:52.343227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:52.344225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:52.345130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:52.345130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:52.345130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:52.345130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 422/837] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:52.352332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b86c6f-712f-4fa5-9c8b-97ffd18c6d1a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:52.476746 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b86c6f-712f-4fa5-9c8b-97ffd18c6d1a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:52.478774 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:52.525182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec83bd-b31a-4f93-87cf-7968321a215b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:52.525868 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec83bd-b31a-4f93-87cf-7968321a215b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.526483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec83bd-b31a-4f93-87cf-7968321a215b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.527934 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec83bd-b31a-4f93-87cf-7968321a215b None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.528629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec83bd-b31a-4f93-87cf-7968321a215b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:52.528931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec83bd-b31a-4f93-87cf-7968321a215b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:52.531267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec83bd-b31a-4f93-87cf-7968321a215b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:52.548175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 416/838] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:52.557125 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fae62-de2b-436c-949c-f978ada60f17 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:52.562478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:52.687884 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fae62-de2b-436c-949c-f978ada60f17 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:52.693945 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62a5fd-576a-4537-ade9-aee8d79efd60 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.694281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62a5fd-576a-4537-ade9-aee8d79efd60 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.694558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62a5fd-576a-4537-ade9-aee8d79efd60 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.696706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 417/839] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:52.702304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33930dc-7025-4395-bb73-4c41bfe02b4a None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.702873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33930dc-7025-4395-bb73-4c41bfe02b4a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.703342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33930dc-7025-4395-bb73-4c41bfe02b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.722513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a33930dc-7025-4395-bb73-4c41bfe02b4a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:52.724066 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a33930dc-7025-4395-bb73-4c41bfe02b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:52.746863 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21308e82-2220-4119-a469-9b1742790b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:52.747548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21308e82-2220-4119-a469-9b1742790b0e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.747827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21308e82-2220-4119-a469-9b1742790b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.748095 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21308e82-2220-4119-a469-9b1742790b0e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.758812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21308e82-2220-4119-a469-9b1742790b0e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:52.760836 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21308e82-2220-4119-a469-9b1742790b0e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-21308e82-2220-4119-a469-9b1742790b0e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:52.775689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:52.776695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:52.777752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:52.777752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:52.777752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:52.777752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:52.777752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:52.777752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:52.777752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:52.777752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:52.777752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:53:52.777752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:52.777752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 423/840] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:53:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:52.783392 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21308e82-2220-4119-a469-9b1742790b0e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:52.810141 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 418/841] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:53:52.863898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:52.909814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21308e82-2220-4119-a469-9b1742790b0e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:52.946617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503642b-3a60-4595-92af-eb38fa31984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:52.947534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503642b-3a60-4595-92af-eb38fa31984f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:52.947988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503642b-3a60-4595-92af-eb38fa31984f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:52.948427 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503642b-3a60-4595-92af-eb38fa31984f None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:52.949331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503642b-3a60-4595-92af-eb38fa31984f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:52.949807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503642b-3a60-4595-92af-eb38fa31984f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:52.951419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503642b-3a60-4595-92af-eb38fa31984f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:52.969399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 424/842] 10.222.0.62 () {56 vars in 1135 bytes} [Fri Sep 9 11:53:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:53:52.974877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33930dc-7025-4395-bb73-4c41bfe02b4a None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:53.067866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:53.077373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33930dc-7025-4395-bb73-4c41bfe02b4a None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:53.099214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503642b-3a60-4595-92af-eb38fa31984f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:53.114839 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce757b-5082-4c4d-866b-62d99740c150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:53.115979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce757b-5082-4c4d-866b-62d99740c150 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:53.116647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce757b-5082-4c4d-866b-62d99740c150 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:53.116925 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce757b-5082-4c4d-866b-62d99740c150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:53.210236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce757b-5082-4c4d-866b-62d99740c150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:53.210236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce757b-5082-4c4d-866b-62d99740c150 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:53.214791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503642b-3a60-4595-92af-eb38fa31984f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:53.253202 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:53.270852 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be36932-36a6-436a-93be-aea13ff973fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:53.271973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be36932-36a6-436a-93be-aea13ff973fc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:53.272645 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be36932-36a6-436a-93be-aea13ff973fc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:53.273121 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be36932-36a6-436a-93be-aea13ff973fc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:53.274419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be36932-36a6-436a-93be-aea13ff973fc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:53.276449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be36932-36a6-436a-93be-aea13ff973fc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-7be36932-36a6-436a-93be-aea13ff973fc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:53.289702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:53.290745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:53.291588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:53.291588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:53.291588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:53.291588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:53.291588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:53.291588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:53.291588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:53.291588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:53.291588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:53.291588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:53.291588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:53:53.291588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:53.291588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:53:53.291588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:53.294368 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 425/843] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:53:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:53.302018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be36932-36a6-436a-93be-aea13ff973fc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:53.337021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:53.348861 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 419/844] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:53:53.355185 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce757b-5082-4c4d-866b-62d99740c150 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:53.428761 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be36932-36a6-436a-93be-aea13ff973fc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:53.468737 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da70f1-3289-47f6-bc51-8f2710529b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:53.469610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da70f1-3289-47f6-bc51-8f2710529b26 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:53.470033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da70f1-3289-47f6-bc51-8f2710529b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:53.470467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da70f1-3289-47f6-bc51-8f2710529b26 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:53.471311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da70f1-3289-47f6-bc51-8f2710529b26 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:53.471833 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da70f1-3289-47f6-bc51-8f2710529b26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:53.473270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce757b-5082-4c4d-866b-62d99740c150 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:53.473631 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da70f1-3289-47f6-bc51-8f2710529b26 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:53.484563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 426/845] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:53:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:53.485772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da70f1-3289-47f6-bc51-8f2710529b26 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:53.512222 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad75b0e-a2d5-4c27-bfb6-20ad92261d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:53.513232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad75b0e-a2d5-4c27-bfb6-20ad92261d6a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:53.513726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad75b0e-a2d5-4c27-bfb6-20ad92261d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:53.514240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad75b0e-a2d5-4c27-bfb6-20ad92261d6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:53.525254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad75b0e-a2d5-4c27-bfb6-20ad92261d6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:53.527068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad75b0e-a2d5-4c27-bfb6-20ad92261d6a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-cad75b0e-a2d5-4c27-bfb6-20ad92261d6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:53.544365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:53.545363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:53.546279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:53.546279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:53.546279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:53.548113 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:53.548851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 420/846] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:53.556761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad75b0e-a2d5-4c27-bfb6-20ad92261d6a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:53.608967 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:53.614252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da70f1-3289-47f6-bc51-8f2710529b26 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:53.657707 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e157c-04bc-40f2-aa5d-f7673ffd7665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:53.657707 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e157c-04bc-40f2-aa5d-f7673ffd7665 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:53.657707 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e157c-04bc-40f2-aa5d-f7673ffd7665 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:53.657707 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e157c-04bc-40f2-aa5d-f7673ffd7665 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:53.658180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e157c-04bc-40f2-aa5d-f7673ffd7665 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:53.658180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e157c-04bc-40f2-aa5d-f7673ffd7665 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:53.658180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e157c-04bc-40f2-aa5d-f7673ffd7665 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:53.666526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 427/847] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:53:53 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:53.670133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad75b0e-a2d5-4c27-bfb6-20ad92261d6a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:53.710003 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbaf3b-53a0-44c5-a26a-8b2a28c8aee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:53.710645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbaf3b-53a0-44c5-a26a-8b2a28c8aee9 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:53.710922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbaf3b-53a0-44c5-a26a-8b2a28c8aee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:53.711182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbaf3b-53a0-44c5-a26a-8b2a28c8aee9 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:53.711829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbaf3b-53a0-44c5-a26a-8b2a28c8aee9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:53.712125 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbaf3b-53a0-44c5-a26a-8b2a28c8aee9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:53.713513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbaf3b-53a0-44c5-a26a-8b2a28c8aee9 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:53.727723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 421/848] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:53.733168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e157c-04bc-40f2-aa5d-f7673ffd7665 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:53.848475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e157c-04bc-40f2-aa5d-f7673ffd7665 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:53.863420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:53.888376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e7541-dc18-40bb-a7f0-f0702b30c9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:53.889039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e7541-dc18-40bb-a7f0-f0702b30c9fa None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:53.889314 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e7541-dc18-40bb-a7f0-f0702b30c9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:53.889583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e7541-dc18-40bb-a7f0-f0702b30c9fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:53.898097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e7541-dc18-40bb-a7f0-f0702b30c9fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:53.899607 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e7541-dc18-40bb-a7f0-f0702b30c9fa None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:53.912688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-906e7541-dc18-40bb-a7f0-f0702b30c9fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:53.912688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:53.912688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:53.912688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:53.912688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:53.912688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:53.912688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:53.912688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:53.912688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:53.912688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:53.912688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:53.912688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:53.912688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:53.912688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:53.912688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:53.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:53.915175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:53.915175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:53.915175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:53.915175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:53.915175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:53.915175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:53.915175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:53.915175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:53.915175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 428/849] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:53.919802 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbaf3b-53a0-44c5-a26a-8b2a28c8aee9 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:54.007950 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc662c8-f01b-4880-9521-a9fc2da53d59 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:54.008554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc662c8-f01b-4880-9521-a9fc2da53d59 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:54.009030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc662c8-f01b-4880-9521-a9fc2da53d59 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:54.012012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 429/850] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:53:54.016457 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2051c72-6547-4778-a1af-28efce53a17f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:54.017020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2051c72-6547-4778-a1af-28efce53a17f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:54.017364 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2051c72-6547-4778-a1af-28efce53a17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:54.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbaf3b-53a0-44c5-a26a-8b2a28c8aee9 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:54.031167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:54.036951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a2051c72-6547-4778-a1af-28efce53a17f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:54.039066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a2051c72-6547-4778-a1af-28efce53a17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:54.073552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfc065-75a0-41ff-8415-6a15d24f4beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:54.074282 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfc065-75a0-41ff-8415-6a15d24f4beb None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:54.074648 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfc065-75a0-41ff-8415-6a15d24f4beb None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:54.074922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfc065-75a0-41ff-8415-6a15d24f4beb None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:54.075625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfc065-75a0-41ff-8415-6a15d24f4beb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:54.075930 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfc065-75a0-41ff-8415-6a15d24f4beb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:54.077366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfc065-75a0-41ff-8415-6a15d24f4beb None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:54.093649 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 422/851] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:54.100538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfc065-75a0-41ff-8415-6a15d24f4beb None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:54.122942 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 430/852] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:53:54.216267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfc065-75a0-41ff-8415-6a15d24f4beb None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:54.254291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ba1ad-93f2-4e64-a178-efcc9751c919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:54.255265 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ba1ad-93f2-4e64-a178-efcc9751c919 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:54.255827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ba1ad-93f2-4e64-a178-efcc9751c919 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:54.256298 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ba1ad-93f2-4e64-a178-efcc9751c919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:54.257283 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ba1ad-93f2-4e64-a178-efcc9751c919 None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=1b2de52919e54bf9bcc352909c9f905f) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:54.257790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ba1ad-93f2-4e64-a178-efcc9751c919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:54.259544 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ba1ad-93f2-4e64-a178-efcc9751c919 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:54.266384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 423/853] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:54 2022] GET /identity/v3/role_assignments?user.id=1b2de52919e54bf9bcc352909c9f905f&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:54.412210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2051c72-6547-4778-a1af-28efce53a17f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:54.524366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2051c72-6547-4778-a1af-28efce53a17f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:54.526900 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:54.560157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3a914-5a00-4e80-ba85-71c901e7e374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:54.560806 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3a914-5a00-4e80-ba85-71c901e7e374 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:54.561084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3a914-5a00-4e80-ba85-71c901e7e374 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:54.561358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3a914-5a00-4e80-ba85-71c901e7e374 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:54.561946 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3a914-5a00-4e80-ba85-71c901e7e374 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:54.563460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3a914-5a00-4e80-ba85-71c901e7e374 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-c4e3a914-5a00-4e80-ba85-71c901e7e374 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:54.573135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:54.574179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:54.575129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:54.575129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:54.575129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:54.575129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:54.575129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:54.575129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:54.575129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:54.575129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:54.575129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:54.575129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:54.575129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:53:54.575129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:54.575129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:53:54.575129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:54.575129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 431/854] 10.222.0.62 () {56 vars in 1132 bytes} [Fri Sep 9 11:53:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:54.584516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ba1ad-93f2-4e64-a178-efcc9751c919 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:54.617251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:54.618979 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe00ec8-605d-4adb-ab6e-b98eef8b2d4b None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:54.620173 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe00ec8-605d-4adb-ab6e-b98eef8b2d4b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:54.620173 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe00ec8-605d-4adb-ab6e-b98eef8b2d4b None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:54.621125 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 432/855] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:54.626317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b40d4-d164-4c08-a0ff-b0bd1b990df4 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:54.626665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b40d4-d164-4c08-a0ff-b0bd1b990df4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:54.626941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b40d4-d164-4c08-a0ff-b0bd1b990df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:54.647260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1e4b40d4-d164-4c08-a0ff-b0bd1b990df4 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:54.648505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1e4b40d4-d164-4c08-a0ff-b0bd1b990df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:54.703445 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ba1ad-93f2-4e64-a178-efcc9751c919 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:54.727853 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 433/856] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:53:54.740988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3be697-afdc-4a35-b8a6-977f57669364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:54.741887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3be697-afdc-4a35-b8a6-977f57669364 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:54.742412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3be697-afdc-4a35-b8a6-977f57669364 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:54.742888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3be697-afdc-4a35-b8a6-977f57669364 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:54.743751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3be697-afdc-4a35-b8a6-977f57669364 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:54.744261 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3be697-afdc-4a35-b8a6-977f57669364 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:54.745825 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3be697-afdc-4a35-b8a6-977f57669364 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:54.751250 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 424/857] 10.222.0.62 () {56 vars in 1126 bytes} [Fri Sep 9 11:53:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:54.758289 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b40d4-d164-4c08-a0ff-b0bd1b990df4 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:54.862534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:54.865426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b40d4-d164-4c08-a0ff-b0bd1b990df4 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:54.901147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c73348-69a7-47e7-a328-5d47fd39c60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:54.901758 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c73348-69a7-47e7-a328-5d47fd39c60e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:54.902032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c73348-69a7-47e7-a328-5d47fd39c60e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:54.902331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c73348-69a7-47e7-a328-5d47fd39c60e None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:54.902962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c73348-69a7-47e7-a328-5d47fd39c60e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:54.903262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c73348-69a7-47e7-a328-5d47fd39c60e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:54.904603 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c73348-69a7-47e7-a328-5d47fd39c60e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:54.909334 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 434/858] 10.222.0.62 () {56 vars in 1126 bytes} [Fri Sep 9 11:53:54 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:55.007189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3be697-afdc-4a35-b8a6-977f57669364 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:55.113417 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:55.124011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3be697-afdc-4a35-b8a6-977f57669364 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:55.159671 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3a531-e197-46d3-9240-3a5c7b9297cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:55.160678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3a531-e197-46d3-9240-3a5c7b9297cd None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:55.161174 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3a531-e197-46d3-9240-3a5c7b9297cd None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:55.161664 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3a531-e197-46d3-9240-3a5c7b9297cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:55.172796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3a531-e197-46d3-9240-3a5c7b9297cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-c6d3a531-e197-46d3-9240-3a5c7b9297cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:53:55.179310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:55.180297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:55.181181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:55.181181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:55.181181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:55.181181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:55.181181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:53:55.181181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:53:55.181181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:53:55.181181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:53:55.181181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:53:55.181181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:53:55.181181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:53:55.181181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:55.183837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 425/859] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:53:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:55.192309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c73348-69a7-47e7-a328-5d47fd39c60e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:55.298412 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c73348-69a7-47e7-a328-5d47fd39c60e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:55.332336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3aaeb-e490-4ea7-bb92-58e291e0edb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:55.333291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3aaeb-e490-4ea7-bb92-58e291e0edb7 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:55.334319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3aaeb-e490-4ea7-bb92-58e291e0edb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:55.334615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3aaeb-e490-4ea7-bb92-58e291e0edb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:55.335274 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3aaeb-e490-4ea7-bb92-58e291e0edb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:55.335572 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3aaeb-e490-4ea7-bb92-58e291e0edb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:55.336920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3aaeb-e490-4ea7-bb92-58e291e0edb7 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:55.345440 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 435/860] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:53:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:55.355231 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3a531-e197-46d3-9240-3a5c7b9297cd None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:55.464298 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3a531-e197-46d3-9240-3a5c7b9297cd None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:55.506680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d510a-106b-49d2-904a-e1a96669d795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:55.507321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d510a-106b-49d2-904a-e1a96669d795 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:55.507948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d510a-106b-49d2-904a-e1a96669d795 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:55.508230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d510a-106b-49d2-904a-e1a96669d795 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:55.519483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d510a-106b-49d2-904a-e1a96669d795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:55.520946 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d510a-106b-49d2-904a-e1a96669d795 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-c81d510a-106b-49d2-904a-e1a96669d795 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:55.535764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:55.536771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:55.538676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:55.538676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:55.538676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:55.538676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 426/861] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:55.544851 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3aaeb-e490-4ea7-bb92-58e291e0edb7 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:55.576623 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:55.659895 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3aaeb-e490-4ea7-bb92-58e291e0edb7 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:55.695101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f59fb1-1594-4056-8b73-814f99b26028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:55.695710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f59fb1-1594-4056-8b73-814f99b26028 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:55.695983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f59fb1-1594-4056-8b73-814f99b26028 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:55.696247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f59fb1-1594-4056-8b73-814f99b26028 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:55.696883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f59fb1-1594-4056-8b73-814f99b26028 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:55.697178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f59fb1-1594-4056-8b73-814f99b26028 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:55.698630 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f59fb1-1594-4056-8b73-814f99b26028 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:55.711729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 436/862] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:55.717277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d510a-106b-49d2-904a-e1a96669d795 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:55.721493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:55.829254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d510a-106b-49d2-904a-e1a96669d795 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:55.864788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bdd8d3-8499-4b61-b07f-7ac7b48d5943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:55.865487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdd8d3-8499-4b61-b07f-7ac7b48d5943 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:55.865767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdd8d3-8499-4b61-b07f-7ac7b48d5943 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:55.866036 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdd8d3-8499-4b61-b07f-7ac7b48d5943 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:55.873983 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bdd8d3-8499-4b61-b07f-7ac7b48d5943 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:55.875454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bdd8d3-8499-4b61-b07f-7ac7b48d5943 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-02bdd8d3-8499-4b61-b07f-7ac7b48d5943 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:53:55.887609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:55.888738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:55.889717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:55.889717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:55.889717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:53:55.889717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:53:55.889717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:53:55.889717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:53:55.889717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:53:55.889717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:53:55.889717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:53:55.889717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:55.889717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 427/863] 10.222.0.62 () {56 vars in 1117 bytes} [Fri Sep 9 11:53:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:55.895237 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f59fb1-1594-4056-8b73-814f99b26028 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:55.987599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:56.008665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f59fb1-1594-4056-8b73-814f99b26028 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:56.049748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9f7bc-f555-4fb3-a327-3cd699e0d382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:56.050400 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9f7bc-f555-4fb3-a327-3cd699e0d382 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:56.050703 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9f7bc-f555-4fb3-a327-3cd699e0d382 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:56.050972 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9f7bc-f555-4fb3-a327-3cd699e0d382 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:56.051698 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9f7bc-f555-4fb3-a327-3cd699e0d382 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:56.051984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9f7bc-f555-4fb3-a327-3cd699e0d382 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:56.054083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9f7bc-f555-4fb3-a327-3cd699e0d382 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:56.070885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b674bf-3926-47e7-a1e0-cc00eec8751c None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:56.071165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b674bf-3926-47e7-a1e0-cc00eec8751c None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:56.071436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b674bf-3926-47e7-a1e0-cc00eec8751c None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:56.073673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 437/865] 10.222.0.62 () {56 vars in 1149 bytes} [Fri Sep 9 11:53:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:56.077064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 428/865] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:56.079095 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b674bf-3926-47e7-a1e0-cc00eec8751c None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:56.079486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec7830-3cb6-4361-956e-e47166e2af17 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:56.079768 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec7830-3cb6-4361-956e-e47166e2af17 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:56.080044 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec7830-3cb6-4361-956e-e47166e2af17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:56.101797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9dec7830-3cb6-4361-956e-e47166e2af17 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:56.103018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9dec7830-3cb6-4361-956e-e47166e2af17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:56.157813 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:56.190246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 438/866] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:53:56.210444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b674bf-3926-47e7-a1e0-cc00eec8751c None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:56.253201 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e34833-4aac-4955-be32-5f069c656116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:56.254319 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e34833-4aac-4955-be32-5f069c656116 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:56.254803 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e34833-4aac-4955-be32-5f069c656116 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:56.255325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e34833-4aac-4955-be32-5f069c656116 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:56.326402 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e34833-4aac-4955-be32-5f069c656116 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:56.328228 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e34833-4aac-4955-be32-5f069c656116 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:56.395100 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:56.432815 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 429/867] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:53:56 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:53:56.438093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec7830-3cb6-4361-956e-e47166e2af17 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:56.492572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e34833-4aac-4955-be32-5f069c656116 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:56.520324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:56.564382 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec7830-3cb6-4361-956e-e47166e2af17 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:56.606241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f262cf2-9431-4147-9296-f4c59626e018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:56.606889 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f262cf2-9431-4147-9296-f4c59626e018 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:56.607163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f262cf2-9431-4147-9296-f4c59626e018 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:56.607348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e34833-4aac-4955-be32-5f069c656116 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:56.607494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f262cf2-9431-4147-9296-f4c59626e018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:56.619219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f262cf2-9431-4147-9296-f4c59626e018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:56.620673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f262cf2-9431-4147-9296-f4c59626e018 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-7f262cf2-9431-4147-9296-f4c59626e018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:53:56.637260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:56.638526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:53:56.639601 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:53:56.639601 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:53:56.639601 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:56.643406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 439/868] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:56.644309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f262cf2-9431-4147-9296-f4c59626e018 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:56.653528 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bc654-55f7-4678-94c2-a9eb72e759b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:56.654315 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bc654-55f7-4678-94c2-a9eb72e759b2 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:56.655272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bc654-55f7-4678-94c2-a9eb72e759b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:56.655637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bc654-55f7-4678-94c2-a9eb72e759b2 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:56.656401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bc654-55f7-4678-94c2-a9eb72e759b2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:56.658348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bc654-55f7-4678-94c2-a9eb72e759b2 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:56.675976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 430/869] 10.222.0.62 () {60 vars in 1141 bytes} [Fri Sep 9 11:53:56 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:53:56.677396 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2072925-8a72-4c80-b7aa-edc8dfcbd4c8 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:56.678340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2072925-8a72-4c80-b7aa-edc8dfcbd4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:56.678697 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2072925-8a72-4c80-b7aa-edc8dfcbd4c8 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:56.682161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 431/870] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:56.685888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67147e87-8b0a-4b7b-8f33-df0b9ed8052b None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:56.686199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67147e87-8b0a-4b7b-8f33-df0b9ed8052b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:56.686469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67147e87-8b0a-4b7b-8f33-df0b9ed8052b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:56.706840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-67147e87-8b0a-4b7b-8f33-df0b9ed8052b None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:56.708209 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-67147e87-8b0a-4b7b-8f33-df0b9ed8052b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:56.729105 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:56.796400 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f262cf2-9431-4147-9296-f4c59626e018 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:56.798878 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 432/871] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:56 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:53:56.823959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:56.836850 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f61b9-d157-4ff5-bf0b-41d16e0d56a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:56.837481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f61b9-d157-4ff5-bf0b-41d16e0d56a4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:56.837760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f61b9-d157-4ff5-bf0b-41d16e0d56a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:56.838033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f61b9-d157-4ff5-bf0b-41d16e0d56a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:56.838735 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f61b9-d157-4ff5-bf0b-41d16e0d56a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:56.839039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f61b9-d157-4ff5-bf0b-41d16e0d56a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:56.840421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f61b9-d157-4ff5-bf0b-41d16e0d56a4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:56.858777 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 440/872] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:56.863325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67147e87-8b0a-4b7b-8f33-df0b9ed8052b None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:56.973092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67147e87-8b0a-4b7b-8f33-df0b9ed8052b None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:57.009334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7990a-15b8-460b-885a-cb34c663670b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:57.009962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7990a-15b8-460b-885a-cb34c663670b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.010266 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7990a-15b8-460b-885a-cb34c663670b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.010647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7990a-15b8-460b-885a-cb34c663670b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.017671 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7990a-15b8-460b-885a-cb34c663670b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:57.019131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7990a-15b8-460b-885a-cb34c663670b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:57.031318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-b3e7990a-15b8-460b-885a-cb34c663670b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:57.031318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:57.031318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:57.031318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:57.031318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:57.031318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:57.031318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:57.031318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:57.031318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:57.031318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:57.031318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:57.031318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:57.031318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:53:57.031318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:53:57.031318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:57.032458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:57.033641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:53:57.033641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:53:57.033641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:53:57.033641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:53:57.033641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:53:57.033641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:53:57.033641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:53:57.033641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:57.033641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 433/873] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:53:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:57.038869 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f61b9-d157-4ff5-bf0b-41d16e0d56a4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:57.080569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7990a-15b8-460b-885a-cb34c663670b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:57.098403 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:57.161709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f61b9-d157-4ff5-bf0b-41d16e0d56a4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:57.206995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d269d-8c67-4d55-9f61-972083941a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:57.207756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d269d-8c67-4d55-9f61-972083941a89 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.208136 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d269d-8c67-4d55-9f61-972083941a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.208576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d269d-8c67-4d55-9f61-972083941a89 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.209557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d269d-8c67-4d55-9f61-972083941a89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:57.209958 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d269d-8c67-4d55-9f61-972083941a89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:57.211695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d269d-8c67-4d55-9f61-972083941a89 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:57.215488 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7990a-15b8-460b-885a-cb34c663670b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:57.226003 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 441/874] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:53:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:57.227122 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c71c6a-62c2-40b5-9b56-8230f4518740 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.227420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c71c6a-62c2-40b5-9b56-8230f4518740 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.227672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c71c6a-62c2-40b5-9b56-8230f4518740 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.234080 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 442/875] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:57.234080 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c71c6a-62c2-40b5-9b56-8230f4518740 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:57.263653 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ee90d-ac83-4037-803a-4434cbcd0b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:57.265102 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ee90d-ac83-4037-803a-4434cbcd0b48 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.265596 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ee90d-ac83-4037-803a-4434cbcd0b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.266036 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ee90d-ac83-4037-803a-4434cbcd0b48 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.266699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ee90d-ac83-4037-803a-4434cbcd0b48 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:57.268073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ee90d-ac83-4037-803a-4434cbcd0b48 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-1a8ee90d-ac83-4037-803a-4434cbcd0b48 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:57.278566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:57.279636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:57.283136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:57.283136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:57.283136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:57.283136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:57.283136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:57.283136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:57.283136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:57.283136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:57.283136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:57.283136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:57.283136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:53:57.283136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:57.283136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:53:57.283136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:53:57.283136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 434/876] 10.222.0.62 () {56 vars in 1126 bytes} [Fri Sep 9 11:53:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:53:57.283952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea43ba-011c-4459-8bc1-90b61ba28e9c None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.283952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea43ba-011c-4459-8bc1-90b61ba28e9c None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.283952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea43ba-011c-4459-8bc1-90b61ba28e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.284913 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:57.301573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-bdea43ba-011c-4459-8bc1-90b61ba28e9c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:57.303481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-bdea43ba-011c-4459-8bc1-90b61ba28e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:57.351951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c71c6a-62c2-40b5-9b56-8230f4518740 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:57.386233 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 435/877] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:57 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:53:57.386233 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea43ba-011c-4459-8bc1-90b61ba28e9c None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:57.390353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c47cf-49fb-4ee7-ac7d-565db35378bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:57.390903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c47cf-49fb-4ee7-ac7d-565db35378bb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.391182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c47cf-49fb-4ee7-ac7d-565db35378bb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.391455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c47cf-49fb-4ee7-ac7d-565db35378bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.392181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c47cf-49fb-4ee7-ac7d-565db35378bb None admin] RBAC: Adding query filter params (role.id=035f7d9f445d406eb2e83ca90fc4d5ca, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=01834ce862ae47cc85668b0a1c66f587) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:57.392489 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c47cf-49fb-4ee7-ac7d-565db35378bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:57.393907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c47cf-49fb-4ee7-ac7d-565db35378bb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:57.395154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:57.401246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 443/878] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:53:57 2022] GET /identity/v3/role_assignments?user.id=01834ce862ae47cc85668b0a1c66f587&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=035f7d9f445d406eb2e83ca90fc4d5ca => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:57.509426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea43ba-011c-4459-8bc1-90b61ba28e9c None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:57.552420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b083f5-a9c0-48f3-9d96-e563c455d0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:57.553567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b083f5-a9c0-48f3-9d96-e563c455d0cf None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.553841 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b083f5-a9c0-48f3-9d96-e563c455d0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.554108 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b083f5-a9c0-48f3-9d96-e563c455d0cf None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.554786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b083f5-a9c0-48f3-9d96-e563c455d0cf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:57.555087 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b083f5-a9c0-48f3-9d96-e563c455d0cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:57.556633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b083f5-a9c0-48f3-9d96-e563c455d0cf None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:57.562226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 436/879] 10.222.0.62 () {56 vars in 1122 bytes} [Fri Sep 9 11:53:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:57.568899 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c47cf-49fb-4ee7-ac7d-565db35378bb None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:57.663785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b083f5-a9c0-48f3-9d96-e563c455d0cf None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:57.680325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c47cf-49fb-4ee7-ac7d-565db35378bb None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:57.690480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:57.729378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290863e-b862-4fd7-aee6-a017f705fd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:57.730285 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290863e-b862-4fd7-aee6-a017f705fd53 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.730744 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290863e-b862-4fd7-aee6-a017f705fd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.731014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290863e-b862-4fd7-aee6-a017f705fd53 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.731709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290863e-b862-4fd7-aee6-a017f705fd53 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:57.732014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290863e-b862-4fd7-aee6-a017f705fd53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:57.733436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290863e-b862-4fd7-aee6-a017f705fd53 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:57.739527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 444/880] 10.222.0.62 () {56 vars in 1122 bytes} [Fri Sep 9 11:53:57 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:57.748976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:57.791702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b083f5-a9c0-48f3-9d96-e563c455d0cf None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:57.834772 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2e5a9-acfa-4897-adef-31c8959ee230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:57.835397 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2e5a9-acfa-4897-adef-31c8959ee230 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:57.835668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2e5a9-acfa-4897-adef-31c8959ee230 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:57.835936 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2e5a9-acfa-4897-adef-31c8959ee230 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:57.836616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2e5a9-acfa-4897-adef-31c8959ee230 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:57.837998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2e5a9-acfa-4897-adef-31c8959ee230 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:57.856814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 437/881] 10.222.0.62 () {60 vars in 1141 bytes} [Fri Sep 9 11:53:57 2022] POST /identity/v3/services => generated 269 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:53:58.993066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221b395-67df-4404-a254-daa0744ff610 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:58.993617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221b395-67df-4404-a254-daa0744ff610 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:58.993617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221b395-67df-4404-a254-daa0744ff610 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:58.995273 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 445/882] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:53:59.002036 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee476b-6c0d-4ec1-8161-7df1609e7143 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:59.002564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee476b-6c0d-4ec1-8161-7df1609e7143 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:59.002657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee476b-6c0d-4ec1-8161-7df1609e7143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:59.021976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-91ee476b-6c0d-4ec1-8161-7df1609e7143 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:59.023215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-91ee476b-6c0d-4ec1-8161-7df1609e7143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:59.106718 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 438/883] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:53:59.389590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221b395-67df-4404-a254-daa0744ff610 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:59.509552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221b395-67df-4404-a254-daa0744ff610 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:59.546324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cd576-483a-438b-abff-f2d14c57f049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:59.546968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cd576-483a-438b-abff-f2d14c57f049 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:59.547248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cd576-483a-438b-abff-f2d14c57f049 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:59.547523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cd576-483a-438b-abff-f2d14c57f049 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:59.548111 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cd576-483a-438b-abff-f2d14c57f049 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:59.549473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cd576-483a-438b-abff-f2d14c57f049 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-325cd576-483a-438b-abff-f2d14c57f049 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:53:59.559433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:53:59.560485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:53:59.561339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:53:59.561339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:53:59.561339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:53:59.561339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:53:59.561339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:53:59.561339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:53:59.561339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:53:59.561339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:53:59.561339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:53:59.561339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:53:59.561339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:53:59.561339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:53:59.561339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:53:59.561339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:53:59.561339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 446/884] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:53:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:53:59.567564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee476b-6c0d-4ec1-8161-7df1609e7143 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:59.631427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:59.682557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee476b-6c0d-4ec1-8161-7df1609e7143 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:59.721189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c75d3-61c5-4b80-81bf-376aa039824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:59.721801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c75d3-61c5-4b80-81bf-376aa039824c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:59.722077 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c75d3-61c5-4b80-81bf-376aa039824c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:59.722368 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c75d3-61c5-4b80-81bf-376aa039824c None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:59.723003 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c75d3-61c5-4b80-81bf-376aa039824c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:59.723299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c75d3-61c5-4b80-81bf-376aa039824c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:59.725301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c75d3-61c5-4b80-81bf-376aa039824c None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:59.730974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 439/885] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:53:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:59.737028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cd576-483a-438b-abff-f2d14c57f049 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:53:59.780721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:53:59.848523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cd576-483a-438b-abff-f2d14c57f049 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:53:59.871511 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b2fdb-579e-4c6a-85fe-24ee91798522 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:59.871511 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b2fdb-579e-4c6a-85fe-24ee91798522 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:59.871511 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b2fdb-579e-4c6a-85fe-24ee91798522 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:59.872823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 440/886] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:53:59.876941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110316cd-076d-4ef3-bb52-7aec67b1d205 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:59.877127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110316cd-076d-4ef3-bb52-7aec67b1d205 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:59.877377 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110316cd-076d-4ef3-bb52-7aec67b1d205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:59.894372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561a17c-6e17-43f1-a7d0-b9203abb5956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:53:59.894372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561a17c-6e17-43f1-a7d0-b9203abb5956 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:53:59.894372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561a17c-6e17-43f1-a7d0-b9203abb5956 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:53:59.894372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561a17c-6e17-43f1-a7d0-b9203abb5956 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:53:59.894862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561a17c-6e17-43f1-a7d0-b9203abb5956 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:53:59.894862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561a17c-6e17-43f1-a7d0-b9203abb5956 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:53:59.894862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561a17c-6e17-43f1-a7d0-b9203abb5956 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:53:59.899681 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 447/887] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:53:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:53:59.902054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-110316cd-076d-4ef3-bb52-7aec67b1d205 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:53:59.903271 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-110316cd-076d-4ef3-bb52-7aec67b1d205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:53:59.905013 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561a17c-6e17-43f1-a7d0-b9203abb5956 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:00.006308 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 441/888] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:53:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:00.029718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:00.033836 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561a17c-6e17-43f1-a7d0-b9203abb5956 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:00.044587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303ad82-e59a-4a1f-a67a-4df8c8d60835 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:00.044906 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303ad82-e59a-4a1f-a67a-4df8c8d60835 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:00.045138 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303ad82-e59a-4a1f-a67a-4df8c8d60835 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:00.046869 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 442/889] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:54:00.052018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8514d-c70a-4e22-ad96-61eb93a40d27 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:00.052302 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8514d-c70a-4e22-ad96-61eb93a40d27 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:00.052630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8514d-c70a-4e22-ad96-61eb93a40d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:00.072145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-15e8514d-c70a-4e22-ad96-61eb93a40d27 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:00.073512 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-15e8514d-c70a-4e22-ad96-61eb93a40d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:00.078709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af1653-c3a8-49f4-8b39-bacea8181d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:00.079324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af1653-c3a8-49f4-8b39-bacea8181d8a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:00.079594 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af1653-c3a8-49f4-8b39-bacea8181d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:00.079862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af1653-c3a8-49f4-8b39-bacea8181d8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:00.080559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af1653-c3a8-49f4-8b39-bacea8181d8a None admin] RBAC: Adding query filter params (interface=public, service_id=5b3e36b196b741c89e2be43651b874fd, region_id=RegionOne) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:00.080923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af1653-c3a8-49f4-8b39-bacea8181d8a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:00.082506 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af1653-c3a8-49f4-8b39-bacea8181d8a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:00.093719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 448/890] 10.222.0.62 () {56 vars in 1251 bytes} [Fri Sep 9 11:53:59 2022] GET /identity/v3/endpoints?service_id=5b3e36b196b741c89e2be43651b874fd&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:00.095089 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af1653-c3a8-49f4-8b39-bacea8181d8a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:00.125948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:00.155815 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 443/891] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:54:00.157157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d3123-1ef0-4bad-912d-324f24d8ca92 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:00.157442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d3123-1ef0-4bad-912d-324f24d8ca92 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:00.157707 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d3123-1ef0-4bad-912d-324f24d8ca92 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:00.159323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 444/892] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:00.172594 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72ec31-867b-4892-962b-181e2d0bd192 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:00.172921 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72ec31-867b-4892-962b-181e2d0bd192 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:00.173363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72ec31-867b-4892-962b-181e2d0bd192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:00.198733 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-fe72ec31-867b-4892-962b-181e2d0bd192 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:00.200088 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-fe72ec31-867b-4892-962b-181e2d0bd192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:00.225074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af1653-c3a8-49f4-8b39-bacea8181d8a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:00.267666 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdded442-e045-4f80-b85d-1d19e86c8f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:00.268223 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdded442-e045-4f80-b85d-1d19e86c8f85 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:00.268503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdded442-e045-4f80-b85d-1d19e86c8f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:00.268966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdded442-e045-4f80-b85d-1d19e86c8f85 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:00.269803 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdded442-e045-4f80-b85d-1d19e86c8f85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:00.271554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdded442-e045-4f80-b85d-1d19e86c8f85 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:00.286158 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 449/893] 10.222.0.62 () {56 vars in 1087 bytes} [Fri Sep 9 11:54:00 2022] GET /identity/v3/services => generated 1268 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:00.296557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 445/894] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:00.300373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdded442-e045-4f80-b85d-1d19e86c8f85 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:00.333887 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:00.421573 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdded442-e045-4f80-b85d-1d19e86c8f85 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:00.434950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72ec31-867b-4892-962b-181e2d0bd192 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:00.468549 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52838eb-d508-45b9-90ea-613503d0b20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:00.469443 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52838eb-d508-45b9-90ea-613503d0b20e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:00.470215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52838eb-d508-45b9-90ea-613503d0b20e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:00.470432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52838eb-d508-45b9-90ea-613503d0b20e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:00.485492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52838eb-d508-45b9-90ea-613503d0b20e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-e52838eb-d508-45b9-90ea-613503d0b20e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:54:00.493200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:00.494416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:00.495453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:00.495453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:00.495453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:00.495453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:00.495453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:54:00.495453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:54:00.495453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:54:00.495453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:54:00.495453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:54:00.495453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:54:00.495453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:54:00.495453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:00.496239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 450/895] 10.222.0.62 () {56 vars in 1120 bytes} [Fri Sep 9 11:54:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:00.502894 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52838eb-d508-45b9-90ea-613503d0b20e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:00.555124 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72ec31-867b-4892-962b-181e2d0bd192 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:00.565316 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:00.592616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f2cb3-ad54-4ba3-bca5-1520a36b4791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:00.593241 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f2cb3-ad54-4ba3-bca5-1520a36b4791 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:00.593859 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f2cb3-ad54-4ba3-bca5-1520a36b4791 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:00.594157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f2cb3-ad54-4ba3-bca5-1520a36b4791 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:00.594831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f2cb3-ad54-4ba3-bca5-1520a36b4791 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:00.596271 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f2cb3-ad54-4ba3-bca5-1520a36b4791 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:00.613553 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:00.617454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 446/896] 10.222.0.62 () {60 vars in 1141 bytes} [Fri Sep 9 11:54:00 2022] POST /identity/v3/services => generated 241 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:54:00.622169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f2cb3-ad54-4ba3-bca5-1520a36b4791 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:00.641335 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52838eb-d508-45b9-90ea-613503d0b20e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:00.698875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f175c9-a68d-46c6-8206-31cc388340aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:00.699856 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f175c9-a68d-46c6-8206-31cc388340aa None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:00.700368 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f175c9-a68d-46c6-8206-31cc388340aa None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:00.700823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f175c9-a68d-46c6-8206-31cc388340aa None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:00.701659 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f175c9-a68d-46c6-8206-31cc388340aa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:00.702145 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f175c9-a68d-46c6-8206-31cc388340aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:00.703694 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f175c9-a68d-46c6-8206-31cc388340aa None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:00.715804 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 451/897] 10.222.0.62 () {56 vars in 1115 bytes} [Fri Sep 9 11:54:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:00.724671 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f175c9-a68d-46c6-8206-31cc388340aa None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:00.747988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f2cb3-ad54-4ba3-bca5-1520a36b4791 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:00.805702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def3009-7a2d-46b1-8e7d-cc8fc6e25c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:00.806335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def3009-7a2d-46b1-8e7d-cc8fc6e25c5a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:00.806872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def3009-7a2d-46b1-8e7d-cc8fc6e25c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:00.807194 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def3009-7a2d-46b1-8e7d-cc8fc6e25c5a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:00.808335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def3009-7a2d-46b1-8e7d-cc8fc6e25c5a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:00.810246 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def3009-7a2d-46b1-8e7d-cc8fc6e25c5a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-2def3009-7a2d-46b1-8e7d-cc8fc6e25c5a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:00.821517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:00.822592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:00.823631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:00.823631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:00.823631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:00.823631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:00.823631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:00.823631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:00.823631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:00.823631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:00.823631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:00.823631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:00.823631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:54:00.823631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:00.823631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:54:00.823631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:00.823631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 447/898] 10.222.0.62 () {56 vars in 1132 bytes} [Fri Sep 9 11:54:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:00.829598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def3009-7a2d-46b1-8e7d-cc8fc6e25c5a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:00.859124 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f175c9-a68d-46c6-8206-31cc388340aa None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:00.866665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:00.901362 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bba405-4e20-4d05-a66c-dcca10ebcdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:00.901995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bba405-4e20-4d05-a66c-dcca10ebcdee None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:00.902296 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bba405-4e20-4d05-a66c-dcca10ebcdee None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:00.902621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bba405-4e20-4d05-a66c-dcca10ebcdee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:00.915248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bba405-4e20-4d05-a66c-dcca10ebcdee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:00.916676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bba405-4e20-4d05-a66c-dcca10ebcdee None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:00.924714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-d0bba405-4e20-4d05-a66c-dcca10ebcdee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:54:00.936011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:00.937027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:54:00.937920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:54:00.937920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:00.937920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:00.937920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 452/899] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:00.943634 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bba405-4e20-4d05-a66c-dcca10ebcdee None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:00.965404 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def3009-7a2d-46b1-8e7d-cc8fc6e25c5a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:01.004088 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3c0c0-b972-44fa-8ff5-8ebfc1137524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:01.004695 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3c0c0-b972-44fa-8ff5-8ebfc1137524 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:01.004973 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3c0c0-b972-44fa-8ff5-8ebfc1137524 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:01.005226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3c0c0-b972-44fa-8ff5-8ebfc1137524 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:01.005894 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3c0c0-b972-44fa-8ff5-8ebfc1137524 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:01.006274 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3c0c0-b972-44fa-8ff5-8ebfc1137524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:01.007760 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3c0c0-b972-44fa-8ff5-8ebfc1137524 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:01.017857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 448/900] 10.222.0.62 () {56 vars in 1126 bytes} [Fri Sep 9 11:54:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:01.018884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3c0c0-b972-44fa-8ff5-8ebfc1137524 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:01.061057 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bba405-4e20-4d05-a66c-dcca10ebcdee None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:01.101009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7eb7a-92b7-454f-aeee-f652a193ac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:01.102000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7eb7a-92b7-454f-aeee-f652a193ac27 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:01.102532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7eb7a-92b7-454f-aeee-f652a193ac27 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:01.103004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7eb7a-92b7-454f-aeee-f652a193ac27 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:01.104288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7eb7a-92b7-454f-aeee-f652a193ac27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:01.104803 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7eb7a-92b7-454f-aeee-f652a193ac27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:01.106741 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7eb7a-92b7-454f-aeee-f652a193ac27 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:01.125007 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 453/901] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:01.130383 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7eb7a-92b7-454f-aeee-f652a193ac27 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:01.133377 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:01.141863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3c0c0-b972-44fa-8ff5-8ebfc1137524 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:01.180560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e229857-e586-4503-8194-ab7b740a80d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:01.180979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e229857-e586-4503-8194-ab7b740a80d2 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:01.181300 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e229857-e586-4503-8194-ab7b740a80d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:01.181567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e229857-e586-4503-8194-ab7b740a80d2 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:01.182234 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e229857-e586-4503-8194-ab7b740a80d2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:01.182533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e229857-e586-4503-8194-ab7b740a80d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:01.184247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e229857-e586-4503-8194-ab7b740a80d2 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:01.190157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 449/902] 10.222.0.62 () {56 vars in 1126 bytes} [Fri Sep 9 11:54:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:01.196861 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e229857-e586-4503-8194-ab7b740a80d2 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:01.202044 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:01.256952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7eb7a-92b7-454f-aeee-f652a193ac27 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:01.296785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408541b-f31f-4404-9b18-de2400fa0551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:01.297946 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408541b-f31f-4404-9b18-de2400fa0551 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:01.298473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408541b-f31f-4404-9b18-de2400fa0551 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:01.298844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408541b-f31f-4404-9b18-de2400fa0551 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:01.307133 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408541b-f31f-4404-9b18-de2400fa0551 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:01.309401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408541b-f31f-4404-9b18-de2400fa0551 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:01.314490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e229857-e586-4503-8194-ab7b740a80d2 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-a408541b-f31f-4404-9b18-de2400fa0551 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:54:01.322593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:01.323532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:01.324482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:01.324482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:01.324482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:54:01.324482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:54:01.324482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:54:01.324482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:54:01.324482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:54:01.324482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:54:01.324482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:54:01.324482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:01.328797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 454/903] 10.222.0.62 () {56 vars in 1117 bytes} [Fri Sep 9 11:54:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:01.329641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408541b-f31f-4404-9b18-de2400fa0551 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:01.356931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72cb12-332e-434d-a232-c774c94e31f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:01.357977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72cb12-332e-434d-a232-c774c94e31f3 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:01.358455 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72cb12-332e-434d-a232-c774c94e31f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:01.359243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72cb12-332e-434d-a232-c774c94e31f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:01.360139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72cb12-332e-434d-a232-c774c94e31f3 None admin] RBAC: Adding query filter params (interface=public, service_id=f2f6f33a40ea4375bc9ac5951cf08a05, region_id=RegionOne) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:01.360668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72cb12-332e-434d-a232-c774c94e31f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:01.362532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72cb12-332e-434d-a232-c774c94e31f3 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:01.373079 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 450/904] 10.222.0.62 () {56 vars in 1251 bytes} [Fri Sep 9 11:54:01 2022] GET /identity/v3/endpoints?service_id=f2f6f33a40ea4375bc9ac5951cf08a05&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:01.374248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72cb12-332e-434d-a232-c774c94e31f3 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:01.403135 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:01.445301 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408541b-f31f-4404-9b18-de2400fa0551 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:01.466974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:01.488015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9478d-f934-4802-8653-7cd4fb9e5a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:01.488925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9478d-f934-4802-8653-7cd4fb9e5a9d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:01.489633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9478d-f934-4802-8653-7cd4fb9e5a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:01.490426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9478d-f934-4802-8653-7cd4fb9e5a9d None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:01.491186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9478d-f934-4802-8653-7cd4fb9e5a9d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:01.491498 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9478d-f934-4802-8653-7cd4fb9e5a9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:01.492933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9478d-f934-4802-8653-7cd4fb9e5a9d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:01.507685 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72cb12-332e-434d-a232-c774c94e31f3 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:01.512558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 455/905] 10.222.0.62 () {56 vars in 1149 bytes} [Fri Sep 9 11:54:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:01.518154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9478d-f934-4802-8653-7cd4fb9e5a9d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:01.551056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119f946-b5aa-485c-83a8-ec3355a17973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:01.551587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119f946-b5aa-485c-83a8-ec3355a17973 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:01.551860 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119f946-b5aa-485c-83a8-ec3355a17973 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:01.552124 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119f946-b5aa-485c-83a8-ec3355a17973 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:01.552761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119f946-b5aa-485c-83a8-ec3355a17973 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:01.554206 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119f946-b5aa-485c-83a8-ec3355a17973 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:01.561945 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 451/906] 10.222.0.62 () {56 vars in 1087 bytes} [Fri Sep 9 11:54:01 2022] GET /identity/v3/services => generated 1497 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:01.651090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9478d-f934-4802-8653-7cd4fb9e5a9d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:01.692795 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65eb68-5350-4fcf-8b1d-a22d3d57c5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:01.693988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65eb68-5350-4fcf-8b1d-a22d3d57c5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:01.694340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65eb68-5350-4fcf-8b1d-a22d3d57c5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:01.694616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65eb68-5350-4fcf-8b1d-a22d3d57c5d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:01.750567 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:01.770447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65eb68-5350-4fcf-8b1d-a22d3d57c5d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:01.772004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65eb68-5350-4fcf-8b1d-a22d3d57c5d4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:01.879790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 456/907] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:54:01 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 11:54:01.885492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119f946-b5aa-485c-83a8-ec3355a17973 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:01.981873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:01.996031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119f946-b5aa-485c-83a8-ec3355a17973 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:02.032055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab77faa-312d-417b-b3c8-c5f33d7c40ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:02.032717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab77faa-312d-417b-b3c8-c5f33d7c40ac None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:02.032964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab77faa-312d-417b-b3c8-c5f33d7c40ac None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:02.033203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab77faa-312d-417b-b3c8-c5f33d7c40ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:02.048246 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab77faa-312d-417b-b3c8-c5f33d7c40ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:02.049715 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab77faa-312d-417b-b3c8-c5f33d7c40ac None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-4ab77faa-312d-417b-b3c8-c5f33d7c40ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:54:02.065907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:02.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:54:02.067808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:54:02.067808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:02.067808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:02.067808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 452/908] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:02.074259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65eb68-5350-4fcf-8b1d-a22d3d57c5d4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:02.157517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:02.194364 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65eb68-5350-4fcf-8b1d-a22d3d57c5d4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:02.229537 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d17cfd-3e91-42a0-94a6-b24fbf46cae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:02.230458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d17cfd-3e91-42a0-94a6-b24fbf46cae0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:02.230952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d17cfd-3e91-42a0-94a6-b24fbf46cae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:02.231419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d17cfd-3e91-42a0-94a6-b24fbf46cae0 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:02.232324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d17cfd-3e91-42a0-94a6-b24fbf46cae0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:02.232834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d17cfd-3e91-42a0-94a6-b24fbf46cae0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:02.234699 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d17cfd-3e91-42a0-94a6-b24fbf46cae0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:02.246809 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 457/909] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:02.252735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab77faa-312d-417b-b3c8-c5f33d7c40ac None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:02.361300 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab77faa-312d-417b-b3c8-c5f33d7c40ac None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:02.397731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b31aa-8861-49e7-8a37-aca5eb3a2bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:02.398783 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b31aa-8861-49e7-8a37-aca5eb3a2bca None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:02.399278 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b31aa-8861-49e7-8a37-aca5eb3a2bca None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:02.399766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b31aa-8861-49e7-8a37-aca5eb3a2bca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:02.407243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b31aa-8861-49e7-8a37-aca5eb3a2bca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:02.409011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b31aa-8861-49e7-8a37-aca5eb3a2bca None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:02.412128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:02.422816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-b60b31aa-8861-49e7-8a37-aca5eb3a2bca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:54:02.422816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:02.422816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:02.422816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:02.422816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:02.422816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:02.422816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:02.422816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:02.422816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:02.422816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:02.422816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:02.422816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:02.422816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:54:02.422816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:54:02.422816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:02.423728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:02.424712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:54:02.424712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:54:02.424712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:54:02.424712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:54:02.424712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:54:02.424712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:54:02.424712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:54:02.424712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:02.427145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 453/910] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:54:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:02.433192 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d17cfd-3e91-42a0-94a6-b24fbf46cae0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:02.551593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d17cfd-3e91-42a0-94a6-b24fbf46cae0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:02.588180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94e5eb-2e2f-4ea2-82bc-54ae5abd808e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:02.588826 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94e5eb-2e2f-4ea2-82bc-54ae5abd808e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:02.589104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94e5eb-2e2f-4ea2-82bc-54ae5abd808e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:02.589387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94e5eb-2e2f-4ea2-82bc-54ae5abd808e None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:02.590157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94e5eb-2e2f-4ea2-82bc-54ae5abd808e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:02.590492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94e5eb-2e2f-4ea2-82bc-54ae5abd808e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:02.592069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94e5eb-2e2f-4ea2-82bc-54ae5abd808e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:02.595311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:02.602936 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761911b5-f829-4ffe-b1ec-8ac7b961bc43 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:02.603252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761911b5-f829-4ffe-b1ec-8ac7b961bc43 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:02.603485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761911b5-f829-4ffe-b1ec-8ac7b961bc43 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:02.605092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 454/911] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:54:02.606457 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 458/912] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:54:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:02.610561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b80d4e-a16e-4782-89be-22ca54bb0a52 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:02.610915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b80d4e-a16e-4782-89be-22ca54bb0a52 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:02.611259 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b80d4e-a16e-4782-89be-22ca54bb0a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:02.617845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94e5eb-2e2f-4ea2-82bc-54ae5abd808e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:02.634786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d8b80d4e-a16e-4782-89be-22ca54bb0a52 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:02.636078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d8b80d4e-a16e-4782-89be-22ca54bb0a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:02.713781 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 455/913] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:02.729500 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94e5eb-2e2f-4ea2-82bc-54ae5abd808e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:02.768336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d21bd0-b89b-4cbe-b062-9f2645be328a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:02.768946 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d21bd0-b89b-4cbe-b062-9f2645be328a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:02.769224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d21bd0-b89b-4cbe-b062-9f2645be328a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:02.769497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d21bd0-b89b-4cbe-b062-9f2645be328a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:02.770306 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d21bd0-b89b-4cbe-b062-9f2645be328a None admin] RBAC: Adding query filter params (role.id=335f4cdc5c87464a8f0aedd8b9831e61, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=01834ce862ae47cc85668b0a1c66f587) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:02.770617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d21bd0-b89b-4cbe-b062-9f2645be328a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:02.772166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d21bd0-b89b-4cbe-b062-9f2645be328a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:02.778573 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 459/914] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:54:02 2022] GET /identity/v3/role_assignments?user.id=01834ce862ae47cc85668b0a1c66f587&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=335f4cdc5c87464a8f0aedd8b9831e61 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:02.987197 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b80d4e-a16e-4782-89be-22ca54bb0a52 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:02.998468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e308f5-cdfe-4741-a80c-351aa11572b4 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:02.999146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e308f5-cdfe-4741-a80c-351aa11572b4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:02.999636 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e308f5-cdfe-4741-a80c-351aa11572b4 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:03.001435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 460/915] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:54:03.007358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2000fe-0b3b-461d-a641-a58e6c35bee7 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:03.007639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2000fe-0b3b-461d-a641-a58e6c35bee7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:03.007909 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2000fe-0b3b-461d-a641-a58e6c35bee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:03.027611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7a2000fe-0b3b-461d-a641-a58e6c35bee7 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:03.029000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7a2000fe-0b3b-461d-a641-a58e6c35bee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:03.102022 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:03.102841 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b80d4e-a16e-4782-89be-22ca54bb0a52 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:03.119854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 461/916] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:03.142992 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:03.151505 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e96a55-a64d-46da-8f9a-65799e43c0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:03.152426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e96a55-a64d-46da-8f9a-65799e43c0df None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:03.152934 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e96a55-a64d-46da-8f9a-65799e43c0df None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:03.153410 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e96a55-a64d-46da-8f9a-65799e43c0df None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:03.154365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e96a55-a64d-46da-8f9a-65799e43c0df None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:03.155987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e96a55-a64d-46da-8f9a-65799e43c0df None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-65e96a55-a64d-46da-8f9a-65799e43c0df None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:03.166092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:03.167110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:03.167941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:03.167941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:03.167941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:03.167941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:03.167941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:03.167941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:03.167941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:03.167941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:03.167941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:03.167941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:03.167941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:54:03.167941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:03.167941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 11:54:03.167941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:03.170687 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 456/917] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:54:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:03.176362 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2000fe-0b3b-461d-a641-a58e6c35bee7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:03.287166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2000fe-0b3b-461d-a641-a58e6c35bee7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:03.323528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18f2ae-167e-4bc3-908f-1d77679bbdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:03.324697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18f2ae-167e-4bc3-908f-1d77679bbdd6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:03.325207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18f2ae-167e-4bc3-908f-1d77679bbdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:03.325697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18f2ae-167e-4bc3-908f-1d77679bbdd6 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:03.326562 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18f2ae-167e-4bc3-908f-1d77679bbdd6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:03.327070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18f2ae-167e-4bc3-908f-1d77679bbdd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:03.328638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18f2ae-167e-4bc3-908f-1d77679bbdd6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:03.334370 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 462/918] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:54:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:03.344284 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e96a55-a64d-46da-8f9a-65799e43c0df None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:03.424465 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18f2ae-167e-4bc3-908f-1d77679bbdd6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:03.466619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e96a55-a64d-46da-8f9a-65799e43c0df None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:03.489410 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:03.514293 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa36c3-32fd-4004-84c1-f3ec4ad4501a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:03.515294 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa36c3-32fd-4004-84c1-f3ec4ad4501a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:03.515774 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa36c3-32fd-4004-84c1-f3ec4ad4501a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:03.516245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa36c3-32fd-4004-84c1-f3ec4ad4501a None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:03.517080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa36c3-32fd-4004-84c1-f3ec4ad4501a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:03.517761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa36c3-32fd-4004-84c1-f3ec4ad4501a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:03.519368 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa36c3-32fd-4004-84c1-f3ec4ad4501a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:03.530696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 457/919] 10.222.0.62 () {56 vars in 1114 bytes} [Fri Sep 9 11:54:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:03.531863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa36c3-32fd-4004-84c1-f3ec4ad4501a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:03.548740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18f2ae-167e-4bc3-908f-1d77679bbdd6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:03.595373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ef528-a88a-4ee6-8cd7-c306460a1923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:03.596041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ef528-a88a-4ee6-8cd7-c306460a1923 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:03.596643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ef528-a88a-4ee6-8cd7-c306460a1923 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:03.596984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ef528-a88a-4ee6-8cd7-c306460a1923 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:03.597854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ef528-a88a-4ee6-8cd7-c306460a1923 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:03.599714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:03.600041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ef528-a88a-4ee6-8cd7-c306460a1923 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-d41ef528-a88a-4ee6-8cd7-c306460a1923 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:03.613187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:03.614315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:03.615219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:03.615219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:03.615219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:03.615219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:03.615219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:03.615219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:03.615219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:03.615219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:03.615219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:03.615219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:03.615219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:54:03.615219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:03.615219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:54:03.615219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:03.618190 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 463/920] 10.222.0.62 () {56 vars in 1126 bytes} [Fri Sep 9 11:54:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:54:03.621548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ef528-a88a-4ee6-8cd7-c306460a1923 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:03.655747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa36c3-32fd-4004-84c1-f3ec4ad4501a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:03.694522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45579d5-a586-459f-9f37-88752c9269d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:03.695562 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45579d5-a586-459f-9f37-88752c9269d1 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:03.696055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45579d5-a586-459f-9f37-88752c9269d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:03.696524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45579d5-a586-459f-9f37-88752c9269d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:03.697393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45579d5-a586-459f-9f37-88752c9269d1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:03.699021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45579d5-a586-459f-9f37-88752c9269d1 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:03.726831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 458/921] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 11:54:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:03.750097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ef528-a88a-4ee6-8cd7-c306460a1923 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:03.786730 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:03.803442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81844f89-dcb6-462c-a795-d20eb7f9ce82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:03.804482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81844f89-dcb6-462c-a795-d20eb7f9ce82 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:03.804992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81844f89-dcb6-462c-a795-d20eb7f9ce82 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:03.805480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81844f89-dcb6-462c-a795-d20eb7f9ce82 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:03.806427 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81844f89-dcb6-462c-a795-d20eb7f9ce82 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:03.806992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81844f89-dcb6-462c-a795-d20eb7f9ce82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:03.808677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81844f89-dcb6-462c-a795-d20eb7f9ce82 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:03.815148 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 464/922] 10.222.0.62 () {56 vars in 1122 bytes} [Fri Sep 9 11:54:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:03.821860 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45579d5-a586-459f-9f37-88752c9269d1 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:03.893614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0aaac7-e66f-4ca6-bcf2-ec6bddaaeef0 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:03.894061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0aaac7-e66f-4ca6-bcf2-ec6bddaaeef0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:03.894456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0aaac7-e66f-4ca6-bcf2-ec6bddaaeef0 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:03.896612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 465/923] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:54:03.902630 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185524e7-4371-44b8-ae2f-3dddad06452c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:03.903018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185524e7-4371-44b8-ae2f-3dddad06452c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:03.903396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185524e7-4371-44b8-ae2f-3dddad06452c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:03.922734 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-185524e7-4371-44b8-ae2f-3dddad06452c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:03.924112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-185524e7-4371-44b8-ae2f-3dddad06452c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:03.963146 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45579d5-a586-459f-9f37-88752c9269d1 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:03.980631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:04.006195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 466/924] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:54:04.011269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6cc24-d31f-49ba-89ce-f7f2c6427042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:04.011892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6cc24-d31f-49ba-89ce-f7f2c6427042 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:04.012174 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6cc24-d31f-49ba-89ce-f7f2c6427042 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:04.012443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6cc24-d31f-49ba-89ce-f7f2c6427042 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:04.013080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6cc24-d31f-49ba-89ce-f7f2c6427042 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:04.013380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6cc24-d31f-49ba-89ce-f7f2c6427042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:04.014978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6cc24-d31f-49ba-89ce-f7f2c6427042 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:04.022230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 459/925] 10.222.0.62 () {56 vars in 1122 bytes} [Fri Sep 9 11:54:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:04.028966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185524e7-4371-44b8-ae2f-3dddad06452c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:04.134976 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185524e7-4371-44b8-ae2f-3dddad06452c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:04.155030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:04.176602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16496ac6-6286-4306-94d7-78058f06bc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:04.178006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16496ac6-6286-4306-94d7-78058f06bc90 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:04.178149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16496ac6-6286-4306-94d7-78058f06bc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:04.178346 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16496ac6-6286-4306-94d7-78058f06bc90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:04.179015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16496ac6-6286-4306-94d7-78058f06bc90 None admin] RBAC: Adding query filter params (interface=public, service_id=3d07a83cfb374ae0ba18bf01d4071e33, region_id=RegionOne) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:04.179525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16496ac6-6286-4306-94d7-78058f06bc90 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:04.180719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16496ac6-6286-4306-94d7-78058f06bc90 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:04.186252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 467/926] 10.222.0.62 () {56 vars in 1251 bytes} [Fri Sep 9 11:54:04 2022] GET /identity/v3/endpoints?service_id=3d07a83cfb374ae0ba18bf01d4071e33&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:04.192341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6cc24-d31f-49ba-89ce-f7f2c6427042 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:04.312258 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6cc24-d31f-49ba-89ce-f7f2c6427042 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:04.325099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16496ac6-6286-4306-94d7-78058f06bc90 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:04.359891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2594689-9018-4c8e-b417-17f386735a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:04.360509 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2594689-9018-4c8e-b417-17f386735a9b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:04.360784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2594689-9018-4c8e-b417-17f386735a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:04.361054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2594689-9018-4c8e-b417-17f386735a9b None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:04.361660 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2594689-9018-4c8e-b417-17f386735a9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:04.363062 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2594689-9018-4c8e-b417-17f386735a9b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:04.370951 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 460/927] 10.222.0.62 () {56 vars in 1087 bytes} [Fri Sep 9 11:54:04 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:04.451444 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16496ac6-6286-4306-94d7-78058f06bc90 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:04.491792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243c88c-a10f-4802-ad94-c484eccd5421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:04.492833 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243c88c-a10f-4802-ad94-c484eccd5421 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:04.493347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243c88c-a10f-4802-ad94-c484eccd5421 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:04.493828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243c88c-a10f-4802-ad94-c484eccd5421 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:04.494671 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243c88c-a10f-4802-ad94-c484eccd5421 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:04.496286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243c88c-a10f-4802-ad94-c484eccd5421 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-6243c88c-a10f-4802-ad94-c484eccd5421 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:04.506918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:04.507941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:04.508839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:04.508839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:04.508839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:04.508839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:04.508839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:04.508839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:04.508839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:04.508839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:04.508839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:04.508839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:04.508839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:54:04.508839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:04.508839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 11:54:04.508839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:04.511826 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 468/928] 10.222.0.62 () {56 vars in 1132 bytes} [Fri Sep 9 11:54:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:54:04.518249 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2594689-9018-4c8e-b417-17f386735a9b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:04.569221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:04.636755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2594689-9018-4c8e-b417-17f386735a9b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:04.673116 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e9ec9-dda0-446d-9465-166fcd326160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:04.673731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e9ec9-dda0-446d-9465-166fcd326160 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:04.674006 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e9ec9-dda0-446d-9465-166fcd326160 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:04.674366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e9ec9-dda0-446d-9465-166fcd326160 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:04.675005 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e9ec9-dda0-446d-9465-166fcd326160 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:04.675355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e9ec9-dda0-446d-9465-166fcd326160 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:04.676740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e9ec9-dda0-446d-9465-166fcd326160 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:04.682290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 461/929] 10.222.0.62 () {56 vars in 1126 bytes} [Fri Sep 9 11:54:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:04.687829 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243c88c-a10f-4802-ad94-c484eccd5421 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:04.727204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:04.801132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243c88c-a10f-4802-ad94-c484eccd5421 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:04.840696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43404c2-6706-4e24-aa83-bd30edf4efc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:04.841682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43404c2-6706-4e24-aa83-bd30edf4efc4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:04.842212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43404c2-6706-4e24-aa83-bd30edf4efc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:04.842687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43404c2-6706-4e24-aa83-bd30edf4efc4 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:04.843548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43404c2-6706-4e24-aa83-bd30edf4efc4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:04.844092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43404c2-6706-4e24-aa83-bd30edf4efc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:04.845719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43404c2-6706-4e24-aa83-bd30edf4efc4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:04.852792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 469/930] 10.222.0.62 () {56 vars in 1126 bytes} [Fri Sep 9 11:54:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:04.859204 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e9ec9-dda0-446d-9465-166fcd326160 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:04.979483 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b63fd2-3a32-4254-92c0-5d2e84dd6814 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:04.979660 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b63fd2-3a32-4254-92c0-5d2e84dd6814 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:04.979941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b63fd2-3a32-4254-92c0-5d2e84dd6814 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:04.981488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 470/931] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:54:04.987459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60eccee-f9f7-4db0-a617-e4352e2313e7 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:04.987783 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60eccee-f9f7-4db0-a617-e4352e2313e7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:04.988012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60eccee-f9f7-4db0-a617-e4352e2313e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:04.994504 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e9ec9-dda0-446d-9465-166fcd326160 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:05.003616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:05.014401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e60eccee-f9f7-4db0-a617-e4352e2313e7 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:05.016774 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e60eccee-f9f7-4db0-a617-e4352e2313e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:05.035634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160a0ef-9b3f-4b7c-9ab5-57e48eda49e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:05.036473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160a0ef-9b3f-4b7c-9ab5-57e48eda49e7 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:05.036473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160a0ef-9b3f-4b7c-9ab5-57e48eda49e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:05.036860 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160a0ef-9b3f-4b7c-9ab5-57e48eda49e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:05.038032 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160a0ef-9b3f-4b7c-9ab5-57e48eda49e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:05.039484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160a0ef-9b3f-4b7c-9ab5-57e48eda49e7 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:05.063800 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 462/932] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 11:54:04 2022] POST /identity/v3/endpoints => generated 347 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:05.118987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 471/933] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:54:05.387810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160a0ef-9b3f-4b7c-9ab5-57e48eda49e7 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:05.496815 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160a0ef-9b3f-4b7c-9ab5-57e48eda49e7 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:05.532131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b9a54-6ef4-4721-99ed-28a94901c565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:05.533193 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b9a54-6ef4-4721-99ed-28a94901c565 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:05.533684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b9a54-6ef4-4721-99ed-28a94901c565 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:05.534181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b9a54-6ef4-4721-99ed-28a94901c565 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:05.545716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b9a54-6ef4-4721-99ed-28a94901c565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-b39b9a54-6ef4-4721-99ed-28a94901c565 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:54:05.551280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:05.552784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:05.553655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:05.553655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:05.553655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:05.553655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:05.553655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:54:05.553655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:54:05.553655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:54:05.553655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:54:05.553655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:54:05.553655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:54:05.553655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:54:05.553655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:05.560919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 463/934] 10.222.0.62 () {56 vars in 1120 bytes} [Fri Sep 9 11:54:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:05.563245 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60eccee-f9f7-4db0-a617-e4352e2313e7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:05.595624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:05.671057 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60eccee-f9f7-4db0-a617-e4352e2313e7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:05.704853 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd279f2-88c4-4db4-82eb-9483902b49c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:05.705467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd279f2-88c4-4db4-82eb-9483902b49c3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:05.705739 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd279f2-88c4-4db4-82eb-9483902b49c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:05.706006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd279f2-88c4-4db4-82eb-9483902b49c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:05.706740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd279f2-88c4-4db4-82eb-9483902b49c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:05.707118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd279f2-88c4-4db4-82eb-9483902b49c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:05.708994 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd279f2-88c4-4db4-82eb-9483902b49c3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:05.719219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 472/935] 10.222.0.62 () {56 vars in 1115 bytes} [Fri Sep 9 11:54:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:05.725636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b9a54-6ef4-4721-99ed-28a94901c565 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:05.752013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:05.836676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b9a54-6ef4-4721-99ed-28a94901c565 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:05.873291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2876ff-838b-4680-9684-e5ff18339f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:05.874391 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2876ff-838b-4680-9684-e5ff18339f06 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:05.874865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2876ff-838b-4680-9684-e5ff18339f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:05.875327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2876ff-838b-4680-9684-e5ff18339f06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:05.887468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2876ff-838b-4680-9684-e5ff18339f06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:05.889251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2876ff-838b-4680-9684-e5ff18339f06 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-9a2876ff-838b-4680-9684-e5ff18339f06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:54:05.904682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:05.905662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:54:05.906580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:54:05.906580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:05.906580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:05.908745 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 464/936] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:54:05.914386 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd279f2-88c4-4db4-82eb-9483902b49c3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:06.009015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:06.026008 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd279f2-88c4-4db4-82eb-9483902b49c3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:06.062705 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28714f-cb12-4711-876b-2a62fb02d306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:06.063311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28714f-cb12-4711-876b-2a62fb02d306 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:06.063584 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28714f-cb12-4711-876b-2a62fb02d306 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:06.063850 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28714f-cb12-4711-876b-2a62fb02d306 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:06.064485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28714f-cb12-4711-876b-2a62fb02d306 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:06.064786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28714f-cb12-4711-876b-2a62fb02d306 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:06.066135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28714f-cb12-4711-876b-2a62fb02d306 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:06.083397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 473/937] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:06.087213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2876ff-838b-4680-9684-e5ff18339f06 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:06.143700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7bd87-9df6-49c9-9b5e-37b87a64a5dc None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:06.143972 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7bd87-9df6-49c9-9b5e-37b87a64a5dc None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:06.144680 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7bd87-9df6-49c9-9b5e-37b87a64a5dc None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:06.146947 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 474/938] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:06.153135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7034951-44eb-417d-bd13-ec76d2b9620e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:06.153401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7034951-44eb-417d-bd13-ec76d2b9620e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:06.153667 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7034951-44eb-417d-bd13-ec76d2b9620e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:06.176547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b7034951-44eb-417d-bd13-ec76d2b9620e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:06.177751 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b7034951-44eb-417d-bd13-ec76d2b9620e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:06.178801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:06.211027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2876ff-838b-4680-9684-e5ff18339f06 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:06.261313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582825c-e4eb-46cb-aff4-1da09dfbe7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:06.262861 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582825c-e4eb-46cb-aff4-1da09dfbe7bc None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:06.263320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582825c-e4eb-46cb-aff4-1da09dfbe7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:06.263744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582825c-e4eb-46cb-aff4-1da09dfbe7bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:06.274148 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 475/939] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:54:06.277349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582825c-e4eb-46cb-aff4-1da09dfbe7bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:06.278852 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582825c-e4eb-46cb-aff4-1da09dfbe7bc None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-f582825c-e4eb-46cb-aff4-1da09dfbe7bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:54:06.290558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:06.291517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:06.292470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:06.292470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:06.292470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:54:06.292470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:54:06.292470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:54:06.292470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:54:06.292470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:54:06.292470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:54:06.292470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:54:06.292470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:06.292470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 465/940] 10.222.0.62 () {56 vars in 1117 bytes} [Fri Sep 9 11:54:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:06.298034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7034951-44eb-417d-bd13-ec76d2b9620e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:06.389115 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:06.412338 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7034951-44eb-417d-bd13-ec76d2b9620e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:06.449338 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e332f8-83f8-4f91-89b6-6e1036dfc27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:06.450017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e332f8-83f8-4f91-89b6-6e1036dfc27b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:06.450577 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e332f8-83f8-4f91-89b6-6e1036dfc27b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:06.450852 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e332f8-83f8-4f91-89b6-6e1036dfc27b None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:06.451589 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e332f8-83f8-4f91-89b6-6e1036dfc27b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:06.451894 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e332f8-83f8-4f91-89b6-6e1036dfc27b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:06.453305 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e332f8-83f8-4f91-89b6-6e1036dfc27b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:06.471795 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 476/941] 10.222.0.62 () {56 vars in 1149 bytes} [Fri Sep 9 11:54:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:06.477535 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582825c-e4eb-46cb-aff4-1da09dfbe7bc None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:06.587006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e332f8-83f8-4f91-89b6-6e1036dfc27b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:06.587747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582825c-e4eb-46cb-aff4-1da09dfbe7bc None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:06.639058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25283609-23dd-42b8-8d4f-0addd2ee8277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:06.640660 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25283609-23dd-42b8-8d4f-0addd2ee8277 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:06.641203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25283609-23dd-42b8-8d4f-0addd2ee8277 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:06.641656 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25283609-23dd-42b8-8d4f-0addd2ee8277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:06.645031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:06.660131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25283609-23dd-42b8-8d4f-0addd2ee8277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:06.663447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25283609-23dd-42b8-8d4f-0addd2ee8277 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-25283609-23dd-42b8-8d4f-0addd2ee8277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:54:06.686473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:06.699662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:54:06.702236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:54:06.702236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:06.702236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:06.702236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 466/942] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:06.704092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25283609-23dd-42b8-8d4f-0addd2ee8277 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:06.720848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e332f8-83f8-4f91-89b6-6e1036dfc27b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:06.760830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fdb8e-7866-4a36-99e3-060f3f386124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:06.761452 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fdb8e-7866-4a36-99e3-060f3f386124 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:06.761737 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fdb8e-7866-4a36-99e3-060f3f386124 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:06.762009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fdb8e-7866-4a36-99e3-060f3f386124 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:06.762702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fdb8e-7866-4a36-99e3-060f3f386124 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:06.764436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fdb8e-7866-4a36-99e3-060f3f386124 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:06.767412 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: WARNING keystone.server.flask.common [None req-f65fdb8e-7866-4a36-99e3-060f3f386124 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 11:54:06.771989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:06.789772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.sql.core [None req-f65fdb8e-7866-4a36-99e3-060f3f386124 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 11:54:06.789772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 11:54:06.789772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [parameters: {'id': '08beb90dfe6f45d8a7d1aea21928e96b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 11:54:06.789772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81448) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 11:54:06.807258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-f65fdb8e-7866-4a36-99e3-060f3f386124 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 08beb90dfe6f45d8a7d1aea21928e96b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 08beb90dfe6f45d8a7d1aea21928e96b. Sep 09 11:54:06.807258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:06.807258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 11:54:06.807258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 11:54:06.807258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 11:54:06.807258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 11:54:06.807258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 11:54:06.807258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 11:54:06.807258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 11:54:06.807258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application conn.query(q) Sep 09 11:54:06.807258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 11:54:06.807258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 11:54:06.807258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application result.read() Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 09 11:54:06.808170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application next(self.gen) Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application yield resource Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application next(self.gen) Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise self.value Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application session.commit() Sep 09 11:54:06.809176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self.session.flush() Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application compat.raise_( Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 11:54:06.810069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 11:54:06.811797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 11:54:06.811797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 11:54:06.811797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 11:54:06.811797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 11:54:06.811797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 11:54:06.811797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 11:54:06.811797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 11:54:06.811797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 11:54:06.811797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 11:54:06.811797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 11:54:06.811797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 11:54:06.811797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 11:54:06.811797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 11:54:06.811797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 11:54:06.811797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application conn.query(q) Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application result.read() Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 11:54:06.812590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 11:54:06.814385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 11:54:06.814385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 11:54:06.814385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 11:54:06.814385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 11:54:06.814385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 11:54:06.814385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 11:54:06.814385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 11:54:06.814385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 11:54:06.814385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [parameters: {'id': '08beb90dfe6f45d8a7d1aea21928e96b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 11:54:06.814385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 11:54:06.814385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:06.814385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:06.814385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:06.814385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:06.814385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:06.816941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:06.819149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:06.819149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 09 11:54:06.819149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 09 11:54:06.819149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:06.819149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:06.819149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 09 11:54:06.819149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 09 11:54:06.819149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 08beb90dfe6f45d8a7d1aea21928e96b. Sep 09 11:54:06.819149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:06.819149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 477/943] 10.222.0.62 () {60 vars in 1140 bytes} [Fri Sep 9 11:54:06 2022] POST /identity/v3/projects => generated 262 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:54:06.819149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723c04a-f235-4cac-bfa0-6c286bda6e46 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:06.819149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723c04a-f235-4cac-bfa0-6c286bda6e46 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:06.820773 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723c04a-f235-4cac-bfa0-6c286bda6e46 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:06.820773 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 478/944] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:06.820773 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723c04a-f235-4cac-bfa0-6c286bda6e46 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:06.836179 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25283609-23dd-42b8-8d4f-0addd2ee8277 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:06.873943 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84209ee8-c27a-44ed-9da8-c12cfa9411ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:06.874495 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84209ee8-c27a-44ed-9da8-c12cfa9411ab None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:06.874766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84209ee8-c27a-44ed-9da8-c12cfa9411ab None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:06.875031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84209ee8-c27a-44ed-9da8-c12cfa9411ab None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:06.875669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84209ee8-c27a-44ed-9da8-c12cfa9411ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:06.875966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84209ee8-c27a-44ed-9da8-c12cfa9411ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:06.877312 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84209ee8-c27a-44ed-9da8-c12cfa9411ab None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:06.895041 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 467/945] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:06.896479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d394368-66b8-49ec-8e71-da3665c79ff9 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:06.896764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d394368-66b8-49ec-8e71-da3665c79ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:06.897041 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d394368-66b8-49ec-8e71-da3665c79ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:06.915649 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-2d394368-66b8-49ec-8e71-da3665c79ff9 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:06.916852 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2d394368-66b8-49ec-8e71-da3665c79ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:06.934006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723c04a-f235-4cac-bfa0-6c286bda6e46 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:06.960750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:06.973080 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98eb4bc-557c-4abe-a57b-37ea23253e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:06.973713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98eb4bc-557c-4abe-a57b-37ea23253e7d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:06.974547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98eb4bc-557c-4abe-a57b-37ea23253e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:06.975269 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98eb4bc-557c-4abe-a57b-37ea23253e7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:06.982314 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98eb4bc-557c-4abe-a57b-37ea23253e7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:06.984119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98eb4bc-557c-4abe-a57b-37ea23253e7d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:06.989059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 468/946] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:06.990417 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d394368-66b8-49ec-8e71-da3665c79ff9 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:07.004471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-b98eb4bc-557c-4abe-a57b-37ea23253e7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:54:07.004471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:07.004471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:07.004471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:07.004471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:07.004471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:07.004471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:07.004471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:07.004471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:07.004471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:07.004471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:07.004471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:07.004471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:54:07.004471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:54:07.004471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:07.005456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:07.006380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:54:07.006380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:54:07.006380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:54:07.006380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:54:07.006380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:54:07.006380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:54:07.006380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:54:07.006380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:07.009052 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 479/947] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:54:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:07.014498 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98eb4bc-557c-4abe-a57b-37ea23253e7d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:07.078944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:07.107427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d394368-66b8-49ec-8e71-da3665c79ff9 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:07.143653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98eb4bc-557c-4abe-a57b-37ea23253e7d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:07.149760 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56c55e-5a12-47b5-9491-131f66e65eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:07.150451 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56c55e-5a12-47b5-9491-131f66e65eee None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:07.150966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56c55e-5a12-47b5-9491-131f66e65eee None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:07.151472 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56c55e-5a12-47b5-9491-131f66e65eee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:07.160778 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56c55e-5a12-47b5-9491-131f66e65eee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:07.162768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56c55e-5a12-47b5-9491-131f66e65eee None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:07.175857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-ff56c55e-5a12-47b5-9491-131f66e65eee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:54:07.175857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:07.175857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:07.175857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:07.175857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:07.175857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:07.175857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:07.175857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:07.175857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:07.175857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:07.175857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:07.175857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:07.175857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:54:07.175857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:54:07.175857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:07.176819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:07.177748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:54:07.177748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:54:07.177748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:54:07.177748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:54:07.177748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:54:07.177748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:54:07.177748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:54:07.177748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:07.181921 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 469/948] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:54:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:07.183217 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56c55e-5a12-47b5-9491-131f66e65eee None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:07.190962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e35800-626f-457a-8f94-cd52b9b7bc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:07.191894 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e35800-626f-457a-8f94-cd52b9b7bc45 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:07.193023 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e35800-626f-457a-8f94-cd52b9b7bc45 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:07.193509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e35800-626f-457a-8f94-cd52b9b7bc45 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:07.194402 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e35800-626f-457a-8f94-cd52b9b7bc45 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:07.194966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e35800-626f-457a-8f94-cd52b9b7bc45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:07.197186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e35800-626f-457a-8f94-cd52b9b7bc45 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:07.221750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 480/949] 10.222.0.62 () {56 vars in 1112 bytes} [Fri Sep 9 11:54:07 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:07.238526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:07.274285 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e35800-626f-457a-8f94-cd52b9b7bc45 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:07.315038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56c55e-5a12-47b5-9491-131f66e65eee None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:07.356373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c0f89-17e9-4c7f-896f-8d96b2e9ba5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:07.357420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c0f89-17e9-4c7f-896f-8d96b2e9ba5e None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:07.357921 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c0f89-17e9-4c7f-896f-8d96b2e9ba5e None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:07.358682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c0f89-17e9-4c7f-896f-8d96b2e9ba5e None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:07.359644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c0f89-17e9-4c7f-896f-8d96b2e9ba5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:07.360056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c0f89-17e9-4c7f-896f-8d96b2e9ba5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:07.361622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c0f89-17e9-4c7f-896f-8d96b2e9ba5e None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:07.375108 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 470/950] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:54:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:07.376558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcf405-9cce-4891-a1f5-8ca7cd45ff31 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:07.376947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcf405-9cce-4891-a1f5-8ca7cd45ff31 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:07.377320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcf405-9cce-4891-a1f5-8ca7cd45ff31 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:07.382162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 471/951] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:07.382162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcf405-9cce-4891-a1f5-8ca7cd45ff31 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:07.405648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e35800-626f-457a-8f94-cd52b9b7bc45 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:07.423376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:07.446726 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf4e24-35ff-4ece-b032-a7bd73d162c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:07.447373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf4e24-35ff-4ece-b032-a7bd73d162c3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:07.447643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf4e24-35ff-4ece-b032-a7bd73d162c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:07.447938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf4e24-35ff-4ece-b032-a7bd73d162c3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:07.448486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf4e24-35ff-4ece-b032-a7bd73d162c3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:07.449840 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf4e24-35ff-4ece-b032-a7bd73d162c3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-45bf4e24-35ff-4ece-b032-a7bd73d162c3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:07.462309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:07.463560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:07.464518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:07.464518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:07.464518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:07.464518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:07.464518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:07.464518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:07.464518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:07.464518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:07.464518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:07.464518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:07.464518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:54:07.464518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:07.464518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:54:07.464518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:07.464518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 481/952] 10.222.0.62 () {56 vars in 1126 bytes} [Fri Sep 9 11:54:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:07.465396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5f6a2-139e-428d-b67f-0f22333d3b16 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:07.465396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5f6a2-139e-428d-b67f-0f22333d3b16 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:07.465396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5f6a2-139e-428d-b67f-0f22333d3b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:07.486686 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d9c5f6a2-139e-428d-b67f-0f22333d3b16 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:07.488711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d9c5f6a2-139e-428d-b67f-0f22333d3b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:07.498070 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:07.507269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcf405-9cce-4891-a1f5-8ca7cd45ff31 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:07.549934 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2599c-994e-4fc4-a2a2-24905a575e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:07.550642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2599c-994e-4fc4-a2a2-24905a575e43 None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:07.550920 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2599c-994e-4fc4-a2a2-24905a575e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:07.551430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2599c-994e-4fc4-a2a2-24905a575e43 None admin] PATH_INFO: `/v3/projects/32b06df0c5504592b012585f193386e7/users/01834ce862ae47cc85668b0a1c66f587/roles/335f4cdc5c87464a8f0aedd8b9831e61` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:07.553448 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2599c-994e-4fc4-a2a2-24905a575e43 None admin] RBAC: Authorizing `identity:create_grant(project_id=32b06df0c5504592b012585f193386e7, user_id=01834ce862ae47cc85668b0a1c66f587, role_id=335f4cdc5c87464a8f0aedd8b9831e61)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:07.554927 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2599c-994e-4fc4-a2a2-24905a575e43 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:07.570714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 482/953] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:07.571430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 472/954] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:54:07 2022] PUT /identity/v3/projects/32b06df0c5504592b012585f193386e7/users/01834ce862ae47cc85668b0a1c66f587/roles/335f4cdc5c87464a8f0aedd8b9831e61 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:07.571695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5f6a2-139e-428d-b67f-0f22333d3b16 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:07.700259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5f6a2-139e-428d-b67f-0f22333d3b16 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:07.709982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:07.742420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a1f21-5c20-4fba-8821-d05257c9c418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:07.743419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a1f21-5c20-4fba-8821-d05257c9c418 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:07.743891 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a1f21-5c20-4fba-8821-d05257c9c418 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:07.744359 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a1f21-5c20-4fba-8821-d05257c9c418 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:07.745188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a1f21-5c20-4fba-8821-d05257c9c418 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:07.745668 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a1f21-5c20-4fba-8821-d05257c9c418 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:07.747252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a1f21-5c20-4fba-8821-d05257c9c418 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:07.753226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 483/955] 10.222.0.62 () {56 vars in 1122 bytes} [Fri Sep 9 11:54:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:07.759244 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2599c-994e-4fc4-a2a2-24905a575e43 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:07.861354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a1f21-5c20-4fba-8821-d05257c9c418 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:07.875870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2599c-994e-4fc4-a2a2-24905a575e43 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:07.915468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a00830-3650-4043-885c-2dc6f653dc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:07.916142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00830-3650-4043-885c-2dc6f653dc5b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:07.916428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00830-3650-4043-885c-2dc6f653dc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:07.916699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00830-3650-4043-885c-2dc6f653dc5b None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:07.917336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a00830-3650-4043-885c-2dc6f653dc5b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:07.917639 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a00830-3650-4043-885c-2dc6f653dc5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:07.919316 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a00830-3650-4043-885c-2dc6f653dc5b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:07.924820 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 473/956] 10.222.0.62 () {56 vars in 1122 bytes} [Fri Sep 9 11:54:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:07.931828 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a00830-3650-4043-885c-2dc6f653dc5b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:07.956541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:07.979492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a1f21-5c20-4fba-8821-d05257c9c418 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:08.025248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a5d51-e76c-4644-82ea-fcf63f2838bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:08.025931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a5d51-e76c-4644-82ea-fcf63f2838bd None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:08.026304 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a5d51-e76c-4644-82ea-fcf63f2838bd None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:08.026587 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a5d51-e76c-4644-82ea-fcf63f2838bd None admin] PATH_INFO: `/v3/services/compute` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:08.027186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a5d51-e76c-4644-82ea-fcf63f2838bd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:08.028554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a5d51-e76c-4644-82ea-fcf63f2838bd None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-a94a5d51-e76c-4644-82ea-fcf63f2838bd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:08.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:08.039130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:08.039988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:08.039988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:08.039988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:08.039988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:08.039988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:08.039988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:08.039988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:08.039988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:08.039988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:08.039988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:08.039988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:54:08.039988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:08.039988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:54:08.039988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:08.044100 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 484/957] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:54:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:54:08.044966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a5d51-e76c-4644-82ea-fcf63f2838bd None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:08.063140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a00830-3650-4043-885c-2dc6f653dc5b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:08.081819 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:08.113625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36ee2e-dace-4cb6-891a-3f279985e8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:08.115331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36ee2e-dace-4cb6-891a-3f279985e8e4 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:08.116052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36ee2e-dace-4cb6-891a-3f279985e8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:08.116940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36ee2e-dace-4cb6-891a-3f279985e8e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:08.118086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36ee2e-dace-4cb6-891a-3f279985e8e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:08.121696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36ee2e-dace-4cb6-891a-3f279985e8e4 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:08.158681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 474/958] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 11:54:07 2022] POST /identity/v3/endpoints => generated 349 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:54:08.165489 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a5d51-e76c-4644-82ea-fcf63f2838bd None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:08.215467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46371805-ad31-49cf-837d-73797dd0cb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:08.216058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46371805-ad31-49cf-837d-73797dd0cb5f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:08.216339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46371805-ad31-49cf-837d-73797dd0cb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:08.216611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46371805-ad31-49cf-837d-73797dd0cb5f None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:08.217243 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46371805-ad31-49cf-837d-73797dd0cb5f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:08.217545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46371805-ad31-49cf-837d-73797dd0cb5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:08.218944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46371805-ad31-49cf-837d-73797dd0cb5f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:08.224980 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 485/959] 10.222.0.62 () {56 vars in 1112 bytes} [Fri Sep 9 11:54:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:08.230862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36ee2e-dace-4cb6-891a-3f279985e8e4 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:08.321633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:08.349575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36ee2e-dace-4cb6-891a-3f279985e8e4 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:08.386921 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a311f-1bde-4dfb-ad36-872657c3ea13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:08.387530 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a311f-1bde-4dfb-ad36-872657c3ea13 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:08.387811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a311f-1bde-4dfb-ad36-872657c3ea13 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:08.388072 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a311f-1bde-4dfb-ad36-872657c3ea13 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:08.388698 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a311f-1bde-4dfb-ad36-872657c3ea13 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:08.389000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a311f-1bde-4dfb-ad36-872657c3ea13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:08.390397 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a311f-1bde-4dfb-ad36-872657c3ea13 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:08.395492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 475/960] 10.222.0.62 () {56 vars in 1112 bytes} [Fri Sep 9 11:54:08 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:09.545947 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fd96a-2893-45c7-a4f8-7e2dea028cf4 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:09.546812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fd96a-2893-45c7-a4f8-7e2dea028cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:09.547236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fd96a-2893-45c7-a4f8-7e2dea028cf4 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:09.549004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 486/961] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:09.554872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd8eb8-b4bc-44c5-a017-5514be3498a6 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:09.555770 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd8eb8-b4bc-44c5-a017-5514be3498a6 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:09.556263 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd8eb8-b4bc-44c5-a017-5514be3498a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:09.577335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-18fd8eb8-b4bc-44c5-a017-5514be3498a6 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:09.578986 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-18fd8eb8-b4bc-44c5-a017-5514be3498a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:09.662219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 476/962] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:54:09.891994 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c5441-d6da-42b7-9834-794d1c77e785 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:09.892336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c5441-d6da-42b7-9834-794d1c77e785 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:09.892542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c5441-d6da-42b7-9834-794d1c77e785 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:09.894174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 487/963] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:09.900034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5386c-7678-4157-8f06-9aee55414c1e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:09.900315 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5386c-7678-4157-8f06-9aee55414c1e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:09.900585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5386c-7678-4157-8f06-9aee55414c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:09.918923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-52d5386c-7678-4157-8f06-9aee55414c1e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:09.920157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-52d5386c-7678-4157-8f06-9aee55414c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:09.951759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c5441-d6da-42b7-9834-794d1c77e785 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:09.991094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:10.001286 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 477/964] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:10.065220 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:10.073218 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c5441-d6da-42b7-9834-794d1c77e785 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:10.108995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954a142-bd68-4547-b10d-71f612d06d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:10.109610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954a142-bd68-4547-b10d-71f612d06d82 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:10.109880 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954a142-bd68-4547-b10d-71f612d06d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:10.110154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954a142-bd68-4547-b10d-71f612d06d82 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:10.110795 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0954a142-bd68-4547-b10d-71f612d06d82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:10.112203 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0954a142-bd68-4547-b10d-71f612d06d82 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:10.127525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.sql.core [None req-0954a142-bd68-4547-b10d-71f612d06d82 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 11:54:10.127525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 11:54:10.127525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [parameters: {'user_id': 'c235f8f84dbb4909899b25f41a92de1c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 11:54:10.127525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81448) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 11:54:10.134610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-0954a142-bd68-4547-b10d-71f612d06d82 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 11:54:10.134610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:10.134610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 11:54:10.134610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 11:54:10.134610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 11:54:10.134610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 11:54:10.134610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 11:54:10.134610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 11:54:10.134610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 11:54:10.134610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application conn.query(q) Sep 09 11:54:10.134610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 11:54:10.134610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 11:54:10.134610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 11:54:10.134610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application result.read() Sep 09 11:54:10.134610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 11:54:10.135494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application next(self.gen) Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application yield resource Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application next(self.gen) Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise self.value Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application session.commit() Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 11:54:10.136482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self.session.flush() Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application compat.raise_( Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 11:54:10.137387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 11:54:10.139139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application conn.query(q) Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application result.read() Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 11:54:10.140196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 11:54:10.141786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 11:54:10.141786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 11:54:10.141786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 11:54:10.141786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 11:54:10.141786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 11:54:10.141786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 11:54:10.141786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c235f8f84dbb4909899b25f41a92de1c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 11:54:10.141786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 11:54:10.141786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:10.141786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:10.141786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:10.141786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:10.141786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:10.141786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:10.141786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 09 11:54:10.143238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 09 11:54:10.152955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 09 11:54:10.152955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 09 11:54:10.152955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 11:54:10.152955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 11:54:10.152955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 11:54:10.152955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:10.152955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 488/965] 10.222.0.62 () {60 vars in 1131 bytes} [Fri Sep 9 11:54:09 2022] POST /identity/v3/users => generated 160 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:10.153897 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5386c-7678-4157-8f06-9aee55414c1e None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:10.264696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5386c-7678-4157-8f06-9aee55414c1e None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:10.300969 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5e068-020c-4290-b8a0-e1e1032340a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:10.301861 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5e068-020c-4290-b8a0-e1e1032340a7 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:10.302789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5e068-020c-4290-b8a0-e1e1032340a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:10.303153 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5e068-020c-4290-b8a0-e1e1032340a7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:10.310654 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5e068-020c-4290-b8a0-e1e1032340a7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:10.312211 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5e068-020c-4290-b8a0-e1e1032340a7 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:10.316347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954a142-bd68-4547-b10d-71f612d06d82 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-31a5e068-020c-4290-b8a0-e1e1032340a7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:54:10.326038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:10.327320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:10.328533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:10.328533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:10.328533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:54:10.328533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:54:10.328533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:54:10.328533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:54:10.328533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:54:10.328533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:54:10.328533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:54:10.328533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:10.335745 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 478/966] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:54:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:10.337143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5e068-020c-4290-b8a0-e1e1032340a7 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:10.405009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:10.443653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954a142-bd68-4547-b10d-71f612d06d82 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:10.451719 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5e068-020c-4290-b8a0-e1e1032340a7 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:10.484665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc6544-6726-4576-9f6c-c718384279f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:10.485312 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc6544-6726-4576-9f6c-c718384279f5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:10.485590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc6544-6726-4576-9f6c-c718384279f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:10.485863 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc6544-6726-4576-9f6c-c718384279f5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:10.487598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131460e-ca12-4e9f-8815-1aaa2202e360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:10.488206 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131460e-ca12-4e9f-8815-1aaa2202e360 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:10.488482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131460e-ca12-4e9f-8815-1aaa2202e360 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:10.488752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131460e-ca12-4e9f-8815-1aaa2202e360 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:10.490176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131460e-ca12-4e9f-8815-1aaa2202e360 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:10.490342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131460e-ca12-4e9f-8815-1aaa2202e360 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:10.491772 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131460e-ca12-4e9f-8815-1aaa2202e360 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:10.499908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc6544-6726-4576-9f6c-c718384279f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-70bc6544-6726-4576-9f6c-c718384279f5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:54:10.505262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:10.506376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:10.507291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:10.507291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:10.507291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:10.507291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:10.507291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:54:10.507291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:54:10.507291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:54:10.507291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:54:10.507291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:54:10.507291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:54:10.507291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:54:10.507291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:10.507291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 489/967] 10.222.0.62 () {56 vars in 1120 bytes} [Fri Sep 9 11:54:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:54:10.508611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e193e-0a6a-440b-a3fa-d0538de8a787 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:10.508723 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e193e-0a6a-440b-a3fa-d0538de8a787 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:10.508949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e193e-0a6a-440b-a3fa-d0538de8a787 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:10.514237 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 490/968] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:10.515180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e193e-0a6a-440b-a3fa-d0538de8a787 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:10.536131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:10.543252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 479/969] 10.222.0.62 () {56 vars in 1101 bytes} [Fri Sep 9 11:54:10 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:10.546171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fb303-4319-4b2a-b2ec-4e4a1ad504a4 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:10.546171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fb303-4319-4b2a-b2ec-4e4a1ad504a4 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:10.546171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fb303-4319-4b2a-b2ec-4e4a1ad504a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:10.572124 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a70fb303-4319-4b2a-b2ec-4e4a1ad504a4 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:10.573947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a70fb303-4319-4b2a-b2ec-4e4a1ad504a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:10.648054 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e193e-0a6a-440b-a3fa-d0538de8a787 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:10.664975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 480/970] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:10.666232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7ad33-a084-448e-a8ce-114e044ce32f None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:10.666511 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7ad33-a084-448e-a8ce-114e044ce32f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:10.666839 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7ad33-a084-448e-a8ce-114e044ce32f None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:10.668436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 481/971] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:10.680448 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a94b7b-3dae-47a4-95fa-a9de3a0e70d9 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:10.681172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a94b7b-3dae-47a4-95fa-a9de3a0e70d9 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:10.681545 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a94b7b-3dae-47a4-95fa-a9de3a0e70d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:10.689647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d80348-6e3d-4a71-9b85-361c19ad0855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:10.690319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d80348-6e3d-4a71-9b85-361c19ad0855 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:10.690602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d80348-6e3d-4a71-9b85-361c19ad0855 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:10.690876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d80348-6e3d-4a71-9b85-361c19ad0855 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:10.691558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d80348-6e3d-4a71-9b85-361c19ad0855 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:10.691863 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d80348-6e3d-4a71-9b85-361c19ad0855 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:10.693933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d80348-6e3d-4a71-9b85-361c19ad0855 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:10.703943 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 491/972] 10.222.0.62 () {56 vars in 1115 bytes} [Fri Sep 9 11:54:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:10.709922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-68a94b7b-3dae-47a4-95fa-a9de3a0e70d9 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:10.710485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d80348-6e3d-4a71-9b85-361c19ad0855 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:10.711259 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-68a94b7b-3dae-47a4-95fa-a9de3a0e70d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:10.771184 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:10.809409 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:10.810221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 482/973] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:54:10.841814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d80348-6e3d-4a71-9b85-361c19ad0855 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:10.888650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d9605-6abf-4a76-a2a9-d372d8282234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:10.889305 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d9605-6abf-4a76-a2a9-d372d8282234 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:10.889587 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d9605-6abf-4a76-a2a9-d372d8282234 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:10.889862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d9605-6abf-4a76-a2a9-d372d8282234 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:10.902613 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d9605-6abf-4a76-a2a9-d372d8282234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:10.904110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d9605-6abf-4a76-a2a9-d372d8282234 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-1a0d9605-6abf-4a76-a2a9-d372d8282234 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:54:10.919448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:10.920448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:54:10.921392 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:54:10.921392 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:10.921392 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:10.921392 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 492/974] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:54:10.928645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a94b7b-3dae-47a4-95fa-a9de3a0e70d9 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:10.954187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d9605-6abf-4a76-a2a9-d372d8282234 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:11.031677 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a94b7b-3dae-47a4-95fa-a9de3a0e70d9 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:11.063240 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d9605-6abf-4a76-a2a9-d372d8282234 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:11.068460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30d467-9911-47d3-8e33-7b416babd19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:11.069290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30d467-9911-47d3-8e33-7b416babd19c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:11.069752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30d467-9911-47d3-8e33-7b416babd19c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:11.070374 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30d467-9911-47d3-8e33-7b416babd19c None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:11.071029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30d467-9911-47d3-8e33-7b416babd19c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:11.071333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30d467-9911-47d3-8e33-7b416babd19c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:11.072826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30d467-9911-47d3-8e33-7b416babd19c None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:11.089552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 483/975] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:11.089552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30d467-9911-47d3-8e33-7b416babd19c None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:11.091770 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:11.097864 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419a9d7-4c34-4736-96d9-daa315120cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:11.098528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419a9d7-4c34-4736-96d9-daa315120cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:11.099349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419a9d7-4c34-4736-96d9-daa315120cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:11.099635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419a9d7-4c34-4736-96d9-daa315120cf6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:11.100221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419a9d7-4c34-4736-96d9-daa315120cf6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:11.101673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419a9d7-4c34-4736-96d9-daa315120cf6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-4419a9d7-4c34-4736-96d9-daa315120cf6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:11.110756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:11.116058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:11.117417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:11.117417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:11.117417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:11.117417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:11.117417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:11.117417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:11.117417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:11.117417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:11.117417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:11.117417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:11.117417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:54:11.117417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:11.117417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:54:11.117417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:11.117417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 493/976] 10.222.0.62 () {56 vars in 1126 bytes} [Fri Sep 9 11:54:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:54:11.118534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419a9d7-4c34-4736-96d9-daa315120cf6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:11.132387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:11.193676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30d467-9911-47d3-8e33-7b416babd19c None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:11.218499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419a9d7-4c34-4736-96d9-daa315120cf6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:11.230692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5c906-9957-46e4-ae35-d39c8ff2abe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:11.231308 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5c906-9957-46e4-ae35-d39c8ff2abe9 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:11.231585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5c906-9957-46e4-ae35-d39c8ff2abe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:11.231853 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5c906-9957-46e4-ae35-d39c8ff2abe9 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:11.232505 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5c906-9957-46e4-ae35-d39c8ff2abe9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:11.234166 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5c906-9957-46e4-ae35-d39c8ff2abe9 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:11.248724 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 484/977] 10.222.0.62 () {60 vars in 1141 bytes} [Fri Sep 9 11:54:11 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:11.249533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5c906-9957-46e4-ae35-d39c8ff2abe9 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:11.261374 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d131f5a-51f2-49b0-953d-1fde9e0bdf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:11.262859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d131f5a-51f2-49b0-953d-1fde9e0bdf50 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:11.263147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d131f5a-51f2-49b0-953d-1fde9e0bdf50 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:11.263760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d131f5a-51f2-49b0-953d-1fde9e0bdf50 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:11.272717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d131f5a-51f2-49b0-953d-1fde9e0bdf50 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:11.274276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d131f5a-51f2-49b0-953d-1fde9e0bdf50 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-3d131f5a-51f2-49b0-953d-1fde9e0bdf50 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:54:11.288197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:11.294523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:11.295516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:11.295516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:11.295516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:54:11.295516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:54:11.295516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:54:11.295516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:54:11.295516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:54:11.295516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:54:11.295516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 11:54:11.295516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:11.295516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 494/978] 10.222.0.62 () {56 vars in 1117 bytes} [Fri Sep 9 11:54:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:11.295516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d131f5a-51f2-49b0-953d-1fde9e0bdf50 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:11.372453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:11.377340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:11.383669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5c906-9957-46e4-ae35-d39c8ff2abe9 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:11.427037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb77038-0844-4958-abe5-e8acc1a54610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:11.427752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb77038-0844-4958-abe5-e8acc1a54610 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:11.428093 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb77038-0844-4958-abe5-e8acc1a54610 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:11.428819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb77038-0844-4958-abe5-e8acc1a54610 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:11.429111 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb77038-0844-4958-abe5-e8acc1a54610 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:11.429998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb77038-0844-4958-abe5-e8acc1a54610 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:11.431519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb77038-0844-4958-abe5-e8acc1a54610 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:11.434873 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d131f5a-51f2-49b0-953d-1fde9e0bdf50 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:11.437479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 485/979] 10.222.0.62 () {56 vars in 1122 bytes} [Fri Sep 9 11:54:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:11.443593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb77038-0844-4958-abe5-e8acc1a54610 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:11.477466 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22debd-c4eb-44b2-b18e-5ce21a7cd91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:11.478003 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22debd-c4eb-44b2-b18e-5ce21a7cd91f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:11.478306 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22debd-c4eb-44b2-b18e-5ce21a7cd91f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:11.478576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22debd-c4eb-44b2-b18e-5ce21a7cd91f None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:11.479310 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22debd-c4eb-44b2-b18e-5ce21a7cd91f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:11.479612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22debd-c4eb-44b2-b18e-5ce21a7cd91f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:11.481032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22debd-c4eb-44b2-b18e-5ce21a7cd91f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:11.499307 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 495/980] 10.222.0.62 () {56 vars in 1149 bytes} [Fri Sep 9 11:54:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:11.505580 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22debd-c4eb-44b2-b18e-5ce21a7cd91f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:11.561314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb77038-0844-4958-abe5-e8acc1a54610 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:11.600648 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd1f77-5695-44ca-887f-4216b6a4f371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:11.601317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd1f77-5695-44ca-887f-4216b6a4f371 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:11.601525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd1f77-5695-44ca-887f-4216b6a4f371 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:11.601797 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd1f77-5695-44ca-887f-4216b6a4f371 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:11.602451 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd1f77-5695-44ca-887f-4216b6a4f371 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:11.602760 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd1f77-5695-44ca-887f-4216b6a4f371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:11.604124 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd1f77-5695-44ca-887f-4216b6a4f371 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:11.609459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 486/981] 10.222.0.62 () {56 vars in 1122 bytes} [Fri Sep 9 11:54:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:11.613072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22debd-c4eb-44b2-b18e-5ce21a7cd91f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:11.615245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd1f77-5695-44ca-887f-4216b6a4f371 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:11.624697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:11.653570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:11.666698 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe70ae-4b8c-4e56-89e1-3aefd793d266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:11.667441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe70ae-4b8c-4e56-89e1-3aefd793d266 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:11.667915 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe70ae-4b8c-4e56-89e1-3aefd793d266 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:11.668371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe70ae-4b8c-4e56-89e1-3aefd793d266 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:11.753161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd1f77-5695-44ca-887f-4216b6a4f371 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:11.759101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe70ae-4b8c-4e56-89e1-3aefd793d266 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:11.760934 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe70ae-4b8c-4e56-89e1-3aefd793d266 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:11.789218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac09638-ebb3-4ffd-a6d8-f6e77e222f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:11.789820 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac09638-ebb3-4ffd-a6d8-f6e77e222f28 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:11.790092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac09638-ebb3-4ffd-a6d8-f6e77e222f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:11.790386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac09638-ebb3-4ffd-a6d8-f6e77e222f28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:11.791094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac09638-ebb3-4ffd-a6d8-f6e77e222f28 None admin] RBAC: Adding query filter params (interface=admin, service_id=3d07a83cfb374ae0ba18bf01d4071e33, region_id=RegionOne) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:11.791395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac09638-ebb3-4ffd-a6d8-f6e77e222f28 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:11.792761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac09638-ebb3-4ffd-a6d8-f6e77e222f28 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:11.803118 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 487/982] 10.222.0.62 () {56 vars in 1249 bytes} [Fri Sep 9 11:54:11 2022] GET /identity/v3/endpoints?service_id=3d07a83cfb374ae0ba18bf01d4071e33&interface=admin®ion_id=RegionOne => generated 193 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:11.804490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac09638-ebb3-4ffd-a6d8-f6e77e222f28 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:11.876479 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 496/983] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:54:11 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 11:54:11.884476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe70ae-4b8c-4e56-89e1-3aefd793d266 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:11.896253 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:11.903453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:11.913806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac09638-ebb3-4ffd-a6d8-f6e77e222f28 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:11.949270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c704da-efc4-4dd9-b1a2-5f6052bb7c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:11.949883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c704da-efc4-4dd9-b1a2-5f6052bb7c81 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:11.950193 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c704da-efc4-4dd9-b1a2-5f6052bb7c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:11.950463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c704da-efc4-4dd9-b1a2-5f6052bb7c81 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:11.951067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c704da-efc4-4dd9-b1a2-5f6052bb7c81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:11.952423 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c704da-efc4-4dd9-b1a2-5f6052bb7c81 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:11.966142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 488/984] 10.222.0.62 () {56 vars in 1087 bytes} [Fri Sep 9 11:54:11 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:54:12.012034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe70ae-4b8c-4e56-89e1-3aefd793d266 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:12.055932 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9a49a-410c-489d-b7c8-09ad9a577da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:12.056597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9a49a-410c-489d-b7c8-09ad9a577da4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:12.056878 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9a49a-410c-489d-b7c8-09ad9a577da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:12.057244 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9a49a-410c-489d-b7c8-09ad9a577da4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:12.071527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9a49a-410c-489d-b7c8-09ad9a577da4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:12.073003 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9a49a-410c-489d-b7c8-09ad9a577da4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-6de9a49a-410c-489d-b7c8-09ad9a577da4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:54:12.089978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:12.091215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:54:12.092109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:54:12.092109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:12.092109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:12.092109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 497/985] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:12.097038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c704da-efc4-4dd9-b1a2-5f6052bb7c81 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:12.211421 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c704da-efc4-4dd9-b1a2-5f6052bb7c81 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:12.250089 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f54b24-b4b8-4604-ba04-c49cd20d9ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:12.250718 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f54b24-b4b8-4604-ba04-c49cd20d9ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:12.250996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f54b24-b4b8-4604-ba04-c49cd20d9ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:12.251267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f54b24-b4b8-4604-ba04-c49cd20d9ce9 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:12.251904 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f54b24-b4b8-4604-ba04-c49cd20d9ce9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:12.252204 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f54b24-b4b8-4604-ba04-c49cd20d9ce9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:12.253556 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f54b24-b4b8-4604-ba04-c49cd20d9ce9 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:12.266075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 489/986] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:12.271667 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9a49a-410c-489d-b7c8-09ad9a577da4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:12.332676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:12.378610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9a49a-410c-489d-b7c8-09ad9a577da4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:12.412470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b2dac-a584-423a-9a3f-d2391c379485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:12.413106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b2dac-a584-423a-9a3f-d2391c379485 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:12.413384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b2dac-a584-423a-9a3f-d2391c379485 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:12.413651 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b2dac-a584-423a-9a3f-d2391c379485 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:12.420295 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b2dac-a584-423a-9a3f-d2391c379485 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:12.421719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b2dac-a584-423a-9a3f-d2391c379485 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:12.433533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-8a9b2dac-a584-423a-9a3f-d2391c379485 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:54:12.433533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:12.433533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:12.433533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:12.433533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:12.433533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:12.433533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:12.433533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:12.433533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:12.433533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:12.433533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:12.433533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:12.433533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:54:12.433533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:54:12.433533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:12.434501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:12.435421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:54:12.435421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:54:12.435421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:54:12.435421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:54:12.435421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:54:12.435421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:54:12.435421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:54:12.435421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:12.439438 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 498/987] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:54:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:12.440818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f54b24-b4b8-4604-ba04-c49cd20d9ce9 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:12.457610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:12.548568 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f54b24-b4b8-4604-ba04-c49cd20d9ce9 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:12.582955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe59ae9-7b1e-46b7-b87c-6be67a11b77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:12.584146 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe59ae9-7b1e-46b7-b87c-6be67a11b77a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:12.584419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe59ae9-7b1e-46b7-b87c-6be67a11b77a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:12.584685 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe59ae9-7b1e-46b7-b87c-6be67a11b77a None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:12.585372 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe59ae9-7b1e-46b7-b87c-6be67a11b77a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:12.585667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe59ae9-7b1e-46b7-b87c-6be67a11b77a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:12.587134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe59ae9-7b1e-46b7-b87c-6be67a11b77a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:12.600038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 490/988] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:54:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:12.605786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b2dac-a584-423a-9a3f-d2391c379485 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:12.708549 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b2dac-a584-423a-9a3f-d2391c379485 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:12.738320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:12.750774 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bc29a-0889-4a07-b8db-efc4745a7e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:12.751387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bc29a-0889-4a07-b8db-efc4745a7e89 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:12.751662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bc29a-0889-4a07-b8db-efc4745a7e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:12.751933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bc29a-0889-4a07-b8db-efc4745a7e89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:12.752658 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6bc29a-0889-4a07-b8db-efc4745a7e89 None admin] RBAC: Adding query filter params (role.id=335f4cdc5c87464a8f0aedd8b9831e61, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=01834ce862ae47cc85668b0a1c66f587) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:12.752965 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6bc29a-0889-4a07-b8db-efc4745a7e89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:12.754402 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6bc29a-0889-4a07-b8db-efc4745a7e89 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:12.762165 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 499/989] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:54:12 2022] GET /identity/v3/role_assignments?user.id=01834ce862ae47cc85668b0a1c66f587&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=335f4cdc5c87464a8f0aedd8b9831e61 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:13.471181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36990ebc-1e47-41da-abc3-12cbb5d06513 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:13.471694 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36990ebc-1e47-41da-abc3-12cbb5d06513 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:13.471694 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36990ebc-1e47-41da-abc3-12cbb5d06513 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:13.473087 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 491/990] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:13.478933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7d66a-2401-4db1-b0d9-302ac989ef71 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:13.479149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7d66a-2401-4db1-b0d9-302ac989ef71 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:13.479420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7d66a-2401-4db1-b0d9-302ac989ef71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:13.498901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-25c7d66a-2401-4db1-b0d9-302ac989ef71 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:13.500404 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-25c7d66a-2401-4db1-b0d9-302ac989ef71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:13.569034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 500/991] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:13.845829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36990ebc-1e47-41da-abc3-12cbb5d06513 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:13.930561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:13.957735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36990ebc-1e47-41da-abc3-12cbb5d06513 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:13.993296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325c92c-4e96-45b5-9298-326e0bb17a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:13.993923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325c92c-4e96-45b5-9298-326e0bb17a64 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:13.994227 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325c92c-4e96-45b5-9298-326e0bb17a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:13.994516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325c92c-4e96-45b5-9298-326e0bb17a64 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:13.995111 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325c92c-4e96-45b5-9298-326e0bb17a64 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:13.996496 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325c92c-4e96-45b5-9298-326e0bb17a64 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-4325c92c-4e96-45b5-9298-326e0bb17a64 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:14.005940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:14.007119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:14.011021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:14.011021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:14.011021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:14.011021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:14.011021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:14.011021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:14.011021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:14.011021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:14.011021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:14.011021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:14.011021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:54:14.011021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:14.011021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:54:14.011021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:14.011021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 492/992] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:54:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:14.013538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7d66a-2401-4db1-b0d9-302ac989ef71 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:14.124933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7d66a-2401-4db1-b0d9-302ac989ef71 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:14.139580 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:14.162231 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b466a-0583-45de-8f2b-19aa38753626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:14.162849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b466a-0583-45de-8f2b-19aa38753626 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:14.163124 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b466a-0583-45de-8f2b-19aa38753626 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:14.163390 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b466a-0583-45de-8f2b-19aa38753626 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:14.164014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b466a-0583-45de-8f2b-19aa38753626 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:14.164313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b466a-0583-45de-8f2b-19aa38753626 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:14.165663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b466a-0583-45de-8f2b-19aa38753626 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:14.175382 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 501/993] 10.222.0.62 () {56 vars in 1112 bytes} [Fri Sep 9 11:54:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:14.176556 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325c92c-4e96-45b5-9298-326e0bb17a64 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:14.189507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70905858-3d93-4133-80ae-ef2d05bd48c4 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:14.190033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70905858-3d93-4133-80ae-ef2d05bd48c4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:14.190033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70905858-3d93-4133-80ae-ef2d05bd48c4 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:14.191685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 502/994] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:14.196936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c230ca-e324-4e6b-a87b-3b25ef0d9299 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:14.197221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c230ca-e324-4e6b-a87b-3b25ef0d9299 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:14.197502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c230ca-e324-4e6b-a87b-3b25ef0d9299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:14.216864 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b6c230ca-e324-4e6b-a87b-3b25ef0d9299 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:14.218255 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b6c230ca-e324-4e6b-a87b-3b25ef0d9299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:14.287736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325c92c-4e96-45b5-9298-326e0bb17a64 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:14.298195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 503/995] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:54:14.326645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449d590-8c21-4154-a5f0-92cb392e17a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:14.327490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449d590-8c21-4154-a5f0-92cb392e17a6 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:14.327912 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449d590-8c21-4154-a5f0-92cb392e17a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:14.328324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449d590-8c21-4154-a5f0-92cb392e17a6 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:14.329108 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449d590-8c21-4154-a5f0-92cb392e17a6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:14.329553 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449d590-8c21-4154-a5f0-92cb392e17a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:14.331122 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449d590-8c21-4154-a5f0-92cb392e17a6 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:14.337467 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 493/996] 10.222.0.62 () {56 vars in 1112 bytes} [Fri Sep 9 11:54:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:14.343129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c230ca-e324-4e6b-a87b-3b25ef0d9299 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:14.448150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c230ca-e324-4e6b-a87b-3b25ef0d9299 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:14.459054 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:14.485338 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09679c17-affc-42fd-91b0-e533fc2bb695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:14.485947 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09679c17-affc-42fd-91b0-e533fc2bb695 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:14.486259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09679c17-affc-42fd-91b0-e533fc2bb695 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:14.486527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09679c17-affc-42fd-91b0-e533fc2bb695 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:14.487213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09679c17-affc-42fd-91b0-e533fc2bb695 None admin] RBAC: Adding query filter params (interface=public, service_id=b86186dbe66c4bdbbae11af12c1d73d2, region_id=RegionOne) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:14.487515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09679c17-affc-42fd-91b0-e533fc2bb695 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:14.488854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09679c17-affc-42fd-91b0-e533fc2bb695 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:14.498442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 504/997] 10.222.0.62 () {56 vars in 1251 bytes} [Fri Sep 9 11:54:14 2022] GET /identity/v3/endpoints?service_id=b86186dbe66c4bdbbae11af12c1d73d2&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:14.499733 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449d590-8c21-4154-a5f0-92cb392e17a6 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:14.502766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:14.568485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09679c17-affc-42fd-91b0-e533fc2bb695 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:14.609151 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449d590-8c21-4154-a5f0-92cb392e17a6 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:14.646508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3406eb-3e68-4b14-92ea-e40237e7c815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:14.647121 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3406eb-3e68-4b14-92ea-e40237e7c815 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:14.647396 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3406eb-3e68-4b14-92ea-e40237e7c815 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:14.647757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3406eb-3e68-4b14-92ea-e40237e7c815 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:14.648372 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3406eb-3e68-4b14-92ea-e40237e7c815 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:14.649740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3406eb-3e68-4b14-92ea-e40237e7c815 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:14.656502 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 494/998] 10.222.0.62 () {56 vars in 1087 bytes} [Fri Sep 9 11:54:14 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:54:14.683780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09679c17-affc-42fd-91b0-e533fc2bb695 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:14.723260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba028ed3-1e11-4226-88d3-f19c2c09af35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:14.724288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba028ed3-1e11-4226-88d3-f19c2c09af35 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:14.724752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba028ed3-1e11-4226-88d3-f19c2c09af35 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:14.725207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba028ed3-1e11-4226-88d3-f19c2c09af35 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:14.725984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba028ed3-1e11-4226-88d3-f19c2c09af35 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:14.727556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba028ed3-1e11-4226-88d3-f19c2c09af35 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-ba028ed3-1e11-4226-88d3-f19c2c09af35 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:14.741425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:14.742876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:14.744586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:14.744586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:14.744586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:14.744586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:14.744586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:14.744586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:14.744586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:14.744586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:14.744586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:14.744586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:14.744586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:54:14.744586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:14.744586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 11:54:14.744586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:14.744586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 505/999] 10.222.0.62 () {56 vars in 1126 bytes} [Fri Sep 9 11:54:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:54:14.750334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3406eb-3e68-4b14-92ea-e40237e7c815 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:14.830950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:14.872548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3406eb-3e68-4b14-92ea-e40237e7c815 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:14.914729 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78e1ee-56ef-431d-8731-319c8ed7a616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:14.915334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78e1ee-56ef-431d-8731-319c8ed7a616 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:14.915605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78e1ee-56ef-431d-8731-319c8ed7a616 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:14.915868 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78e1ee-56ef-431d-8731-319c8ed7a616 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:14.916493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78e1ee-56ef-431d-8731-319c8ed7a616 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:14.916795 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78e1ee-56ef-431d-8731-319c8ed7a616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:14.918167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78e1ee-56ef-431d-8731-319c8ed7a616 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:14.923114 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 495/1000] 10.222.0.62 () {56 vars in 1122 bytes} [Fri Sep 9 11:54:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:14.929503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba028ed3-1e11-4226-88d3-f19c2c09af35 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:14.970503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:15.041544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba028ed3-1e11-4226-88d3-f19c2c09af35 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:15.084840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7008b-850b-44e7-9263-04bd53057ad1 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.085164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7008b-850b-44e7-9263-04bd53057ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.085379 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7008b-850b-44e7-9263-04bd53057ad1 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.086907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 496/1001] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:15.091686 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca7dc4-4808-48e2-aebb-e467e16bb289 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.091976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca7dc4-4808-48e2-aebb-e467e16bb289 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.092253 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca7dc4-4808-48e2-aebb-e467e16bb289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.092890 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461ff6b-71f5-48bc-8cd3-bb2af1c79692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:15.093510 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461ff6b-71f5-48bc-8cd3-bb2af1c79692 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.093787 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461ff6b-71f5-48bc-8cd3-bb2af1c79692 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.094056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461ff6b-71f5-48bc-8cd3-bb2af1c79692 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.094779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461ff6b-71f5-48bc-8cd3-bb2af1c79692 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:15.095185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461ff6b-71f5-48bc-8cd3-bb2af1c79692 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:15.096758 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461ff6b-71f5-48bc-8cd3-bb2af1c79692 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:15.103890 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 506/1002] 10.222.0.62 () {56 vars in 1122 bytes} [Fri Sep 9 11:54:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:15.108977 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461ff6b-71f5-48bc-8cd3-bb2af1c79692 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:15.118226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-0dca7dc4-4808-48e2-aebb-e467e16bb289 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:15.118552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0dca7dc4-4808-48e2-aebb-e467e16bb289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:15.197495 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 497/1003] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:54:15.217325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:15.222871 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461ff6b-71f5-48bc-8cd3-bb2af1c79692 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:15.258141 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65add5-cad5-4173-80c1-b0535b93bed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:15.258741 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65add5-cad5-4173-80c1-b0535b93bed7 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.259013 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65add5-cad5-4173-80c1-b0535b93bed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.259280 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65add5-cad5-4173-80c1-b0535b93bed7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.259935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65add5-cad5-4173-80c1-b0535b93bed7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:15.261274 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65add5-cad5-4173-80c1-b0535b93bed7 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:15.280819 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 507/1004] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 11:54:15 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:54:15.465063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca7dc4-4808-48e2-aebb-e467e16bb289 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:15.533211 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:15.580357 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca7dc4-4808-48e2-aebb-e467e16bb289 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:15.622053 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adef2f-24ad-447c-88d7-fe9378c936c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:15.622717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adef2f-24ad-447c-88d7-fe9378c936c0 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.622996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adef2f-24ad-447c-88d7-fe9378c936c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.623269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adef2f-24ad-447c-88d7-fe9378c936c0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.623849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adef2f-24ad-447c-88d7-fe9378c936c0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:15.625212 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adef2f-24ad-447c-88d7-fe9378c936c0 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-56adef2f-24ad-447c-88d7-fe9378c936c0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:15.634343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:15.635424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:15.636314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:15.636314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:15.636314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:15.636314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:15.636314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:15.636314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:15.636314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:15.636314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:15.636314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:15.636314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:15.636314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:54:15.636314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:15.636314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:54:15.636314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:15.636314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 498/1005] 10.222.0.62 () {56 vars in 1105 bytes} [Fri Sep 9 11:54:15 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:15.643077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65add5-cad5-4173-80c1-b0535b93bed7 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:15.751959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65add5-cad5-4173-80c1-b0535b93bed7 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:15.790106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8f28c-761a-4134-afda-d3283d50ca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:15.790744 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8f28c-761a-4134-afda-d3283d50ca9b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.791024 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8f28c-761a-4134-afda-d3283d50ca9b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.791295 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8f28c-761a-4134-afda-d3283d50ca9b None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.793281 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d8f28c-761a-4134-afda-d3283d50ca9b None admin] RBAC: Adding query filter params (name=image) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:15.793281 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d8f28c-761a-4134-afda-d3283d50ca9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:15.793596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d8f28c-761a-4134-afda-d3283d50ca9b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:15.803566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 508/1006] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:15.804950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adef2f-24ad-447c-88d7-fe9378c936c0 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:15.909725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adef2f-24ad-447c-88d7-fe9378c936c0 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:15.930865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:15.946804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a91aea-da5d-47f3-b4a1-7534e2f74b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:15.947739 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a91aea-da5d-47f3-b4a1-7534e2f74b7b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:15.948015 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a91aea-da5d-47f3-b4a1-7534e2f74b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:15.948281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a91aea-da5d-47f3-b4a1-7534e2f74b7b None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:15.948911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a91aea-da5d-47f3-b4a1-7534e2f74b7b None admin] RBAC: Adding query filter params (type=image) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:15.949213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a91aea-da5d-47f3-b4a1-7534e2f74b7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:15.950617 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a91aea-da5d-47f3-b4a1-7534e2f74b7b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:15.962151 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 499/1007] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:15 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:16.845022 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b65a32-d28a-417f-874b-1e1872f931a3 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:16.845549 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b65a32-d28a-417f-874b-1e1872f931a3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:16.845549 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b65a32-d28a-417f-874b-1e1872f931a3 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:16.847078 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 509/1008] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:16.852598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240a161-2256-4622-b40d-cc404264c8aa None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:16.852988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240a161-2256-4622-b40d-cc404264c8aa None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:16.853206 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240a161-2256-4622-b40d-cc404264c8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:16.874573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-8240a161-2256-4622-b40d-cc404264c8aa None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:16.875757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-8240a161-2256-4622-b40d-cc404264c8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:16.947508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 500/1009] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:16 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:17.211877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b65a32-d28a-417f-874b-1e1872f931a3 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:17.240084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:17.319068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b65a32-d28a-417f-874b-1e1872f931a3 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:17.353250 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7b036-d736-4f1c-a171-c0570c94fd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:17.353897 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7b036-d736-4f1c-a171-c0570c94fd23 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:17.354202 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7b036-d736-4f1c-a171-c0570c94fd23 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:17.354475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7b036-d736-4f1c-a171-c0570c94fd23 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:17.355055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be7b036-d736-4f1c-a171-c0570c94fd23 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:17.356410 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be7b036-d736-4f1c-a171-c0570c94fd23 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-4be7b036-d736-4f1c-a171-c0570c94fd23 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:17.365449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:17.366476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:17.367340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:17.367340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:17.367340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:17.367340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:17.367340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:17.367340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:17.367340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:17.367340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:17.367340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:17.367340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:17.367340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:54:17.367340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:17.367340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 11:54:17.367340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:17.368714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 510/1010] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:54:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:17.372356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240a161-2256-4622-b40d-cc404264c8aa None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:17.498004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240a161-2256-4622-b40d-cc404264c8aa None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:17.508649 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bae5e-e858-4005-959f-c399854b1b0f None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:17.508962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bae5e-e858-4005-959f-c399854b1b0f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:17.509208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bae5e-e858-4005-959f-c399854b1b0f None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:17.510610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 511/1011] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:17.513380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:17.515950 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dc9b6-6c6f-47bc-9ba3-abd1429ec256 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:17.516513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dc9b6-6c6f-47bc-9ba3-abd1429ec256 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:17.516513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dc9b6-6c6f-47bc-9ba3-abd1429ec256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:17.536932 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-0e4dc9b6-6c6f-47bc-9ba3-abd1429ec256 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:17.538843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0e4dc9b6-6c6f-47bc-9ba3-abd1429ec256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:17.543028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3c3d6-9af1-4144-b1d8-4c5a6a9f4816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:17.543641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3c3d6-9af1-4144-b1d8-4c5a6a9f4816 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:17.543913 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3c3d6-9af1-4144-b1d8-4c5a6a9f4816 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:17.544177 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3c3d6-9af1-4144-b1d8-4c5a6a9f4816 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:17.544804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3c3d6-9af1-4144-b1d8-4c5a6a9f4816 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:17.545100 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3c3d6-9af1-4144-b1d8-4c5a6a9f4816 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:17.546484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3c3d6-9af1-4144-b1d8-4c5a6a9f4816 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:17.555840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 501/1012] 10.222.0.62 () {56 vars in 1112 bytes} [Fri Sep 9 11:54:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:17.557027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3c3d6-9af1-4144-b1d8-4c5a6a9f4816 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:17.620687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 512/1013] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:17 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:17.660741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3c3d6-9af1-4144-b1d8-4c5a6a9f4816 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:17.695317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfa2a6-d33c-4c4a-8ed0-0b3c0852d109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:17.695920 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfa2a6-d33c-4c4a-8ed0-0b3c0852d109 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:17.696184 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfa2a6-d33c-4c4a-8ed0-0b3c0852d109 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:17.696450 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfa2a6-d33c-4c4a-8ed0-0b3c0852d109 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:17.697078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfa2a6-d33c-4c4a-8ed0-0b3c0852d109 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:17.697384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfa2a6-d33c-4c4a-8ed0-0b3c0852d109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:17.698755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfa2a6-d33c-4c4a-8ed0-0b3c0852d109 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:17.703806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 502/1014] 10.222.0.62 () {56 vars in 1112 bytes} [Fri Sep 9 11:54:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:17.709275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dc9b6-6c6f-47bc-9ba3-abd1429ec256 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:17.728009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:17.813378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dc9b6-6c6f-47bc-9ba3-abd1429ec256 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:17.848215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cd1e4-4754-4b62-8587-6275a313e29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:17.848962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cd1e4-4754-4b62-8587-6275a313e29c None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:17.849225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cd1e4-4754-4b62-8587-6275a313e29c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:17.849452 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cd1e4-4754-4b62-8587-6275a313e29c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:17.850153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cd1e4-4754-4b62-8587-6275a313e29c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:17.851501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cd1e4-4754-4b62-8587-6275a313e29c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:17.878149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 513/1015] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 11:54:17 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:17.900606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfa2a6-d33c-4c4a-8ed0-0b3c0852d109 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:17.955896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:18.022031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfa2a6-d33c-4c4a-8ed0-0b3c0852d109 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:18.070515 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b93a2-99b3-4232-a101-09cd34e81f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:18.071084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b93a2-99b3-4232-a101-09cd34e81f82 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:18.071363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b93a2-99b3-4232-a101-09cd34e81f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:18.071631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b93a2-99b3-4232-a101-09cd34e81f82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:18.072746 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b93a2-99b3-4232-a101-09cd34e81f82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:18.074217 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b93a2-99b3-4232-a101-09cd34e81f82 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:18.075999 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 503/1016] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:18.082099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cd1e4-4754-4b62-8587-6275a313e29c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:18.157091 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19541c2-049a-4d81-9704-2bb230fd66a7 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:18.157370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19541c2-049a-4d81-9704-2bb230fd66a7 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:18.157679 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19541c2-049a-4d81-9704-2bb230fd66a7 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:18.159435 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 504/1017] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:54:18.173097 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb501fb-2fd1-47e3-9552-2ce01d9ed3ee None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:18.173552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb501fb-2fd1-47e3-9552-2ce01d9ed3ee None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:18.173858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb501fb-2fd1-47e3-9552-2ce01d9ed3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:18.200492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-efb501fb-2fd1-47e3-9552-2ce01d9ed3ee None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:18.203378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-efb501fb-2fd1-47e3-9552-2ce01d9ed3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:18.208918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:18.257507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cd1e4-4754-4b62-8587-6275a313e29c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:18.303662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 505/1018] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:18 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:18.305578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42187f57-747e-4435-b093-6bb429363cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:18.306869 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42187f57-747e-4435-b093-6bb429363cd9 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:18.307333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42187f57-747e-4435-b093-6bb429363cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:18.307602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42187f57-747e-4435-b093-6bb429363cd9 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:18.308290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42187f57-747e-4435-b093-6bb429363cd9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:18.309794 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42187f57-747e-4435-b093-6bb429363cd9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:18.344484 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 514/1019] 10.222.0.62 () {60 vars in 1141 bytes} [Fri Sep 9 11:54:18 2022] POST /identity/v3/projects => generated 311 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:18.572228 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb501fb-2fd1-47e3-9552-2ce01d9ed3ee None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:18.621420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:18.704436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb501fb-2fd1-47e3-9552-2ce01d9ed3ee None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:18.755593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bd001-c713-468d-a751-fcce611eb285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:18.756382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bd001-c713-468d-a751-fcce611eb285 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:18.756807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bd001-c713-468d-a751-fcce611eb285 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:18.757092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bd001-c713-468d-a751-fcce611eb285 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:18.757987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765bd001-c713-468d-a751-fcce611eb285 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:18.760047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765bd001-c713-468d-a751-fcce611eb285 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:18.780464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 506/1020] 10.222.0.62 () {60 vars in 1141 bytes} [Fri Sep 9 11:54:18 2022] POST /identity/v3/services => generated 242 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:20.560959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac5cc4-f96d-4fd9-9158-0231765b4a75 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:20.561483 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac5cc4-f96d-4fd9-9158-0231765b4a75 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:20.561483 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac5cc4-f96d-4fd9-9158-0231765b4a75 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:20.562797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 515/1021] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:20.570418 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e726e6-b6c1-4089-b8c7-c6a06433d60e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:20.571219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e726e6-b6c1-4089-b8c7-c6a06433d60e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:20.571358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e726e6-b6c1-4089-b8c7-c6a06433d60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:20.584831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f27a8a-38ed-4e5c-ae2f-9483d85db054 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:20.585227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f27a8a-38ed-4e5c-ae2f-9483d85db054 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:20.585609 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f27a8a-38ed-4e5c-ae2f-9483d85db054 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:20.587705 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 516/1022] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:20.590235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-88e726e6-b6c1-4089-b8c7-c6a06433d60e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:20.592278 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-88e726e6-b6c1-4089-b8c7-c6a06433d60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:20.593733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba15bd-c21b-49d1-95ff-dd3ab2ac7f55 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:20.594020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba15bd-c21b-49d1-95ff-dd3ab2ac7f55 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:20.594332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba15bd-c21b-49d1-95ff-dd3ab2ac7f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:20.624067 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-68ba15bd-c21b-49d1-95ff-dd3ab2ac7f55 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:20.625265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-68ba15bd-c21b-49d1-95ff-dd3ab2ac7f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:20.685985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 507/1023] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:54:20.730166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 517/1024] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:20.971696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e726e6-b6c1-4089-b8c7-c6a06433d60e None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:20.998598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:21.001735 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa693df0-f865-46cb-816d-d811dcd18568 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:21.002073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa693df0-f865-46cb-816d-d811dcd18568 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:21.003378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa693df0-f865-46cb-816d-d811dcd18568 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:21.005669 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 518/1025] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:21.011427 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60fcde-c6aa-49d1-8339-40bf2f6e3d95 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:21.011885 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60fcde-c6aa-49d1-8339-40bf2f6e3d95 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:21.012366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60fcde-c6aa-49d1-8339-40bf2f6e3d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:21.034251 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4f60fcde-c6aa-49d1-8339-40bf2f6e3d95 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:21.035448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4f60fcde-c6aa-49d1-8339-40bf2f6e3d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:21.084079 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e726e6-b6c1-4089-b8c7-c6a06433d60e None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:21.097308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:21.118527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 519/1026] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:54:21.118740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54f01a-511f-436e-8688-d11e080675ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:21.119371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60fcde-c6aa-49d1-8339-40bf2f6e3d95 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:21.119492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54f01a-511f-436e-8688-d11e080675ad None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:21.119650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54f01a-511f-436e-8688-d11e080675ad None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:21.119911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54f01a-511f-436e-8688-d11e080675ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:21.121132 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54f01a-511f-436e-8688-d11e080675ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:21.122599 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54f01a-511f-436e-8688-d11e080675ad None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:21.124393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 508/1027] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:21.130454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54f01a-511f-436e-8688-d11e080675ad None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:21.245320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:21.257163 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54f01a-511f-436e-8688-d11e080675ad None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:21.257674 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60fcde-c6aa-49d1-8339-40bf2f6e3d95 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:21.293998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f344b-0e6c-4b1a-8ce4-4d6aa1b7e521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:21.294659 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f344b-0e6c-4b1a-8ce4-4d6aa1b7e521 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:21.295003 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f344b-0e6c-4b1a-8ce4-4d6aa1b7e521 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:21.295231 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f344b-0e6c-4b1a-8ce4-4d6aa1b7e521 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:21.295858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f344b-0e6c-4b1a-8ce4-4d6aa1b7e521 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:21.297247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae6d58-a0cd-4562-a689-921e9113fee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:21.297384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f344b-0e6c-4b1a-8ce4-4d6aa1b7e521 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:21.298252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae6d58-a0cd-4562-a689-921e9113fee8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:21.298691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae6d58-a0cd-4562-a689-921e9113fee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:21.299123 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae6d58-a0cd-4562-a689-921e9113fee8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:21.311441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae6d58-a0cd-4562-a689-921e9113fee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-2aae6d58-a0cd-4562-a689-921e9113fee8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:54:21.316791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:21.318366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:21.324685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:21.324685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:21.324685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:21.324685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:21.324685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:54:21.324685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:54:21.324685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:54:21.324685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:54:21.324685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:54:21.324685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:54:21.324685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:54:21.324685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:21.324685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 520/1028] 10.222.0.62 () {56 vars in 1120 bytes} [Fri Sep 9 11:54:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:21.325579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 509/1029] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:54:21 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:54:21.331328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae6d58-a0cd-4562-a689-921e9113fee8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:21.392183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:21.402055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f344b-0e6c-4b1a-8ce4-4d6aa1b7e521 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:21.466099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae6d58-a0cd-4562-a689-921e9113fee8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:21.511354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9737d-3697-4447-8726-8f6acbe5676b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:21.513549 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9737d-3697-4447-8726-8f6acbe5676b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:21.513866 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9737d-3697-4447-8726-8f6acbe5676b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:21.514191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9737d-3697-4447-8726-8f6acbe5676b None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:21.514890 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9737d-3697-4447-8726-8f6acbe5676b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:21.515190 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9737d-3697-4447-8726-8f6acbe5676b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:21.516617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9737d-3697-4447-8726-8f6acbe5676b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:21.527152 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f344b-0e6c-4b1a-8ce4-4d6aa1b7e521 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:21.537811 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 521/1030] 10.222.0.62 () {56 vars in 1115 bytes} [Fri Sep 9 11:54:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:21.544321 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9737d-3697-4447-8726-8f6acbe5676b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:21.570962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece20390-8cff-434f-bd96-125ea7fc68ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:21.571605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece20390-8cff-434f-bd96-125ea7fc68ac None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:21.571881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece20390-8cff-434f-bd96-125ea7fc68ac None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:21.572185 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece20390-8cff-434f-bd96-125ea7fc68ac None admin] PATH_INFO: `/v3/services/image` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:21.572744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece20390-8cff-434f-bd96-125ea7fc68ac None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:21.574267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece20390-8cff-434f-bd96-125ea7fc68ac None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-ece20390-8cff-434f-bd96-125ea7fc68ac None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:21.585964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:21.589240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:21.591788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:21.591788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:21.591788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:21.591788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:21.591788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:21.591788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:21.591788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:21.591788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:21.591788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:21.591788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:21.591788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:54:21.591788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:21.591788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:54:21.591788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:21.591788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 510/1031] 10.222.0.62 () {56 vars in 1105 bytes} [Fri Sep 9 11:54:21 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:21.594766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece20390-8cff-434f-bd96-125ea7fc68ac None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:21.613154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:21.692079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9737d-3697-4447-8726-8f6acbe5676b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:21.704137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:21.733291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317a5e4-0970-4053-a474-93b44928055e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:21.734212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317a5e4-0970-4053-a474-93b44928055e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:21.734519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317a5e4-0970-4053-a474-93b44928055e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:21.734930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317a5e4-0970-4053-a474-93b44928055e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:21.741360 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece20390-8cff-434f-bd96-125ea7fc68ac None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:21.748678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317a5e4-0970-4053-a474-93b44928055e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:21.750733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317a5e4-0970-4053-a474-93b44928055e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-e317a5e4-0970-4053-a474-93b44928055e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:54:21.769060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:21.770098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:54:21.771168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:54:21.771168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:21.771168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:21.775330 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 522/1032] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:54:21.776288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317a5e4-0970-4053-a474-93b44928055e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:21.798260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c507b-035a-4126-83ce-92c51614cd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:21.798884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c507b-035a-4126-83ce-92c51614cd94 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:21.799158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c507b-035a-4126-83ce-92c51614cd94 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:21.799429 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c507b-035a-4126-83ce-92c51614cd94 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:21.800131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c507b-035a-4126-83ce-92c51614cd94 None admin] RBAC: Adding query filter params (name=image) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:21.800437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c507b-035a-4126-83ce-92c51614cd94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:21.802644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c507b-035a-4126-83ce-92c51614cd94 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:21.810307 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 511/1033] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:21.814872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c507b-035a-4126-83ce-92c51614cd94 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:21.897889 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:21.905519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317a5e4-0970-4053-a474-93b44928055e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:21.927557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c507b-035a-4126-83ce-92c51614cd94 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:21.949061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2416f-9621-4487-a585-ea3e0260dd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:21.949834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2416f-9621-4487-a585-ea3e0260dd8e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:21.950137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2416f-9621-4487-a585-ea3e0260dd8e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:21.950410 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2416f-9621-4487-a585-ea3e0260dd8e None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:21.951059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2416f-9621-4487-a585-ea3e0260dd8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:21.951363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2416f-9621-4487-a585-ea3e0260dd8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:21.952721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2416f-9621-4487-a585-ea3e0260dd8e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:21.966314 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 523/1034] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:21.969625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6704e-446e-423b-8765-edba892e2101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:21.970267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6704e-446e-423b-8765-edba892e2101 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:21.970545 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6704e-446e-423b-8765-edba892e2101 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:21.970955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6704e-446e-423b-8765-edba892e2101 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:21.971693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6704e-446e-423b-8765-edba892e2101 None admin] RBAC: Adding query filter params (type=image) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:21.971833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6704e-446e-423b-8765-edba892e2101 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:21.972145 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2416f-9621-4487-a585-ea3e0260dd8e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:21.973176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6704e-446e-423b-8765-edba892e2101 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:21.983908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 512/1035] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:21.985809 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6704e-446e-423b-8765-edba892e2101 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:21.998477 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:22.086448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2416f-9621-4487-a585-ea3e0260dd8e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:22.098750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6704e-446e-423b-8765-edba892e2101 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:22.123358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e9569-d2c3-4c69-99d8-b18a9fe7d54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:22.124006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e9569-d2c3-4c69-99d8-b18a9fe7d54e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:22.124284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e9569-d2c3-4c69-99d8-b18a9fe7d54e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:22.124556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e9569-d2c3-4c69-99d8-b18a9fe7d54e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:22.132643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e9569-d2c3-4c69-99d8-b18a9fe7d54e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:22.134070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e9569-d2c3-4c69-99d8-b18a9fe7d54e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:22.139832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094b56e-ced7-4a51-8618-1070eafa2b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:22.140720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094b56e-ced7-4a51-8618-1070eafa2b10 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:22.141164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094b56e-ced7-4a51-8618-1070eafa2b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:22.141577 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094b56e-ced7-4a51-8618-1070eafa2b10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:22.142454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094b56e-ced7-4a51-8618-1070eafa2b10 None admin] RBAC: Adding query filter params (interface=public, service_id=1034cacb82ec41a58cdf8215e557d9f1, region_id=RegionOne) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:22.142906 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094b56e-ced7-4a51-8618-1070eafa2b10 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:22.144428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094b56e-ced7-4a51-8618-1070eafa2b10 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-e64e9569-d2c3-4c69-99d8-b18a9fe7d54e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:54:22.146593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:22.147709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:22.148701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:22.148701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:22.148701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:54:22.148701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:54:22.148701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:54:22.148701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:54:22.148701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:54:22.148701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:54:22.148701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:54:22.148701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:22.148701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 524/1036] 10.222.0.62 () {56 vars in 1093 bytes} [Fri Sep 9 11:54:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:22.153702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e9569-d2c3-4c69-99d8-b18a9fe7d54e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:22.157662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 513/1037] 10.222.0.62 () {56 vars in 1251 bytes} [Fri Sep 9 11:54:21 2022] GET /identity/v3/endpoints?service_id=1034cacb82ec41a58cdf8215e557d9f1&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:22.163956 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094b56e-ced7-4a51-8618-1070eafa2b10 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:22.166953 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:22.253219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:22.276056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e9569-d2c3-4c69-99d8-b18a9fe7d54e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:22.278726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094b56e-ced7-4a51-8618-1070eafa2b10 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:22.309898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba9130-340a-4e9f-9d45-b3d4582b3c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:22.310853 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba9130-340a-4e9f-9d45-b3d4582b3c76 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:22.311322 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba9130-340a-4e9f-9d45-b3d4582b3c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:22.311596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba9130-340a-4e9f-9d45-b3d4582b3c76 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:22.312321 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba9130-340a-4e9f-9d45-b3d4582b3c76 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:22.312624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba9130-340a-4e9f-9d45-b3d4582b3c76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:22.314039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba9130-340a-4e9f-9d45-b3d4582b3c76 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:22.316003 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad6da8-5aae-4bbe-b619-4aab7a53d53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:22.316719 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad6da8-5aae-4bbe-b619-4aab7a53d53f None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:22.317024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad6da8-5aae-4bbe-b619-4aab7a53d53f None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:22.317868 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad6da8-5aae-4bbe-b619-4aab7a53d53f None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:22.318493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad6da8-5aae-4bbe-b619-4aab7a53d53f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:22.319852 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad6da8-5aae-4bbe-b619-4aab7a53d53f None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:22.338241 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 514/1038] 10.222.0.62 () {56 vars in 1087 bytes} [Fri Sep 9 11:54:22 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 11:54:22.342831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 525/1039] 10.222.0.62 () {56 vars in 1133 bytes} [Fri Sep 9 11:54:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:22.349154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad6da8-5aae-4bbe-b619-4aab7a53d53f None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:22.424225 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:22.463836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad6da8-5aae-4bbe-b619-4aab7a53d53f None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:22.512764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8256fa4-058b-4a8b-b7b7-ef796f8eb911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:22.513367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8256fa4-058b-4a8b-b7b7-ef796f8eb911 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:22.513648 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8256fa4-058b-4a8b-b7b7-ef796f8eb911 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:22.513920 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8256fa4-058b-4a8b-b7b7-ef796f8eb911 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:22.587169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8256fa4-058b-4a8b-b7b7-ef796f8eb911 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:22.588708 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8256fa4-058b-4a8b-b7b7-ef796f8eb911 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:22.654156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:22.696770 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 515/1040] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:54:22 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:54:22.702308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba9130-340a-4e9f-9d45-b3d4582b3c76 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:22.813652 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba9130-340a-4e9f-9d45-b3d4582b3c76 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:22.850947 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d344b8-0802-469d-86da-c644ceb5bb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:22.851567 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d344b8-0802-469d-86da-c644ceb5bb35 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:22.851844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d344b8-0802-469d-86da-c644ceb5bb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:22.852131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d344b8-0802-469d-86da-c644ceb5bb35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:22.859247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:22.864000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d344b8-0802-469d-86da-c644ceb5bb35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:22.865534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d344b8-0802-469d-86da-c644ceb5bb35 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-d7d344b8-0802-469d-86da-c644ceb5bb35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:54:22.881327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:22.882458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:54:22.883363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:54:22.883363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:22.883363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:22.886246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 526/1041] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:22.888412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8256fa4-058b-4a8b-b7b7-ef796f8eb911 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:22.991893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8256fa4-058b-4a8b-b7b7-ef796f8eb911 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:23.026984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11299b97-aee3-42ac-8ca5-751cf065d913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:23.027590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11299b97-aee3-42ac-8ca5-751cf065d913 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:23.027881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11299b97-aee3-42ac-8ca5-751cf065d913 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:23.028135 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11299b97-aee3-42ac-8ca5-751cf065d913 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:23.028778 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11299b97-aee3-42ac-8ca5-751cf065d913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:23.029083 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11299b97-aee3-42ac-8ca5-751cf065d913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:23.030470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11299b97-aee3-42ac-8ca5-751cf065d913 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:23.042650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 516/1042] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:23.048026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d344b8-0802-469d-86da-c644ceb5bb35 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:23.163807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d344b8-0802-469d-86da-c644ceb5bb35 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:23.211195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86537405-eec9-42cc-a34a-56e5397b8618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:23.211849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86537405-eec9-42cc-a34a-56e5397b8618 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:23.212119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86537405-eec9-42cc-a34a-56e5397b8618 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:23.212388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86537405-eec9-42cc-a34a-56e5397b8618 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:23.222964 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86537405-eec9-42cc-a34a-56e5397b8618 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:23.225439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86537405-eec9-42cc-a34a-56e5397b8618 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:23.238091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-86537405-eec9-42cc-a34a-56e5397b8618 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:54:23.238091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:23.238091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:23.238091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:23.238091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:23.238091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:23.238091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:23.238091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:23.238091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:23.238091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:23.238091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:23.238091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:23.238091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:54:23.238091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:54:23.238091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:23.239632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:23.240557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:54:23.240557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:54:23.240557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:54:23.240557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:54:23.240557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:54:23.240557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:54:23.240557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:54:23.240557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:23.240557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 527/1043] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:54:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:23.248603 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11299b97-aee3-42ac-8ca5-751cf065d913 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:23.281061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:23.359566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11299b97-aee3-42ac-8ca5-751cf065d913 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:23.394221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015f734-2d5b-4da6-b1ed-87cb752ae624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:23.394829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015f734-2d5b-4da6-b1ed-87cb752ae624 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:23.395186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015f734-2d5b-4da6-b1ed-87cb752ae624 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:23.395457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015f734-2d5b-4da6-b1ed-87cb752ae624 None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:23.396165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015f734-2d5b-4da6-b1ed-87cb752ae624 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:23.396467 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015f734-2d5b-4da6-b1ed-87cb752ae624 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:23.397981 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015f734-2d5b-4da6-b1ed-87cb752ae624 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:23.416996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 517/1044] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:54:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:23.418450 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86537405-eec9-42cc-a34a-56e5397b8618 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:23.470989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:23.527480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86537405-eec9-42cc-a34a-56e5397b8618 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:23.565589 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec38cb-75a9-435d-9f7f-fccd4fbc0a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:23.566257 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec38cb-75a9-435d-9f7f-fccd4fbc0a83 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:23.566537 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec38cb-75a9-435d-9f7f-fccd4fbc0a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:23.566805 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec38cb-75a9-435d-9f7f-fccd4fbc0a83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:23.568088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec38cb-75a9-435d-9f7f-fccd4fbc0a83 None admin] RBAC: Adding query filter params (role.id=335f4cdc5c87464a8f0aedd8b9831e61, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=1b30e80c421245b5a1ed5849b28ea90e) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:23.568394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec38cb-75a9-435d-9f7f-fccd4fbc0a83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:23.569932 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec38cb-75a9-435d-9f7f-fccd4fbc0a83 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:23.582201 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 528/1045] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:54:23 2022] GET /identity/v3/role_assignments?user.id=1b30e80c421245b5a1ed5849b28ea90e&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=335f4cdc5c87464a8f0aedd8b9831e61 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:23.682101 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba79a73-acaf-437b-999e-c9b204d20136 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:23.682706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba79a73-acaf-437b-999e-c9b204d20136 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:23.682706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba79a73-acaf-437b-999e-c9b204d20136 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:23.684112 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 518/1046] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:23.689708 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945d9d9-33f5-4fdd-b81f-5207906650a2 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:23.690001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945d9d9-33f5-4fdd-b81f-5207906650a2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:23.690302 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945d9d9-33f5-4fdd-b81f-5207906650a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:23.710204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e945d9d9-33f5-4fdd-b81f-5207906650a2 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:23.711434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e945d9d9-33f5-4fdd-b81f-5207906650a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:23.785245 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 529/1047] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:24.053331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba79a73-acaf-437b-999e-c9b204d20136 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:24.142588 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:24.160484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba79a73-acaf-437b-999e-c9b204d20136 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:24.203055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ef527-20e0-46d7-ac0e-94e09109b521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:24.203696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ef527-20e0-46d7-ac0e-94e09109b521 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:24.203973 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ef527-20e0-46d7-ac0e-94e09109b521 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:24.204243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ef527-20e0-46d7-ac0e-94e09109b521 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:24.216760 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ef527-20e0-46d7-ac0e-94e09109b521 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-c15ef527-20e0-46d7-ac0e-94e09109b521 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:54:24.222776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:24.223740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:24.224598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:24.224598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:24.224598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:24.224598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:24.224598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:54:24.224598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:54:24.224598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:54:24.224598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:54:24.224598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:54:24.224598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:54:24.224598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:54:24.224598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:24.224598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 519/1048] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:54:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:24.231930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945d9d9-33f5-4fdd-b81f-5207906650a2 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:24.279131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:24.337129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945d9d9-33f5-4fdd-b81f-5207906650a2 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:24.372261 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866365ce-fe49-433d-83d5-9ebef4caee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:24.372872 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866365ce-fe49-433d-83d5-9ebef4caee23 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:24.373147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866365ce-fe49-433d-83d5-9ebef4caee23 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:24.373414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866365ce-fe49-433d-83d5-9ebef4caee23 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:24.374075 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866365ce-fe49-433d-83d5-9ebef4caee23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:24.374405 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866365ce-fe49-433d-83d5-9ebef4caee23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:24.375759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866365ce-fe49-433d-83d5-9ebef4caee23 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:24.384801 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 530/1049] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:54:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:24.390451 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ef527-20e0-46d7-ac0e-94e09109b521 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:24.501047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ef527-20e0-46d7-ac0e-94e09109b521 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:24.535634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0097e7-ba47-4113-af27-5b2ac24e8a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:24.536266 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0097e7-ba47-4113-af27-5b2ac24e8a7a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:24.536541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0097e7-ba47-4113-af27-5b2ac24e8a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:24.536807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0097e7-ba47-4113-af27-5b2ac24e8a7a None admin] PATH_INFO: `/v3/users/0ac00601bdaa4f7c8b6b165f4bc47715` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:24.550664 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0097e7-ba47-4113-af27-5b2ac24e8a7a None admin] RBAC: Authorizing `identity:get_user(user_id=0ac00601bdaa4f7c8b6b165f4bc47715)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:24.552641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0097e7-ba47-4113-af27-5b2ac24e8a7a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:24.554523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 520/1050] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:54:24 2022] GET /identity/v3/users/0ac00601bdaa4f7c8b6b165f4bc47715 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:24.560141 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866365ce-fe49-433d-83d5-9ebef4caee23 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:24.562848 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497e638-2502-4984-9f5c-791c8f0f35d1 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:24.563161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497e638-2502-4984-9f5c-791c8f0f35d1 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:24.563401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497e638-2502-4984-9f5c-791c8f0f35d1 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:24.564730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 521/1051] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:54:24.570892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3c579-9b80-400b-8712-ebaaedaf9f3a None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:24.571131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3c579-9b80-400b-8712-ebaaedaf9f3a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:24.571382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3c579-9b80-400b-8712-ebaaedaf9f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:24.580520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:24.594342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-c3a3c579-9b80-400b-8712-ebaaedaf9f3a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:24.595463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-c3a3c579-9b80-400b-8712-ebaaedaf9f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:24.668467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866365ce-fe49-433d-83d5-9ebef4caee23 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:24.671118 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 522/1052] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:54:24.679154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:24.708504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1cd10-822a-4036-883e-6434b42ed6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:24.709108 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1cd10-822a-4036-883e-6434b42ed6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:24.709380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1cd10-822a-4036-883e-6434b42ed6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:24.709643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1cd10-822a-4036-883e-6434b42ed6e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:24.777197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1cd10-822a-4036-883e-6434b42ed6e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:24.778853 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1cd10-822a-4036-883e-6434b42ed6e9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:24.882247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 531/1053] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:54:24 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:54:24.887830 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3c579-9b80-400b-8712-ebaaedaf9f3a None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:24.945485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1cd10-822a-4036-883e-6434b42ed6e9 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:24.954021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:24.992411 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3c579-9b80-400b-8712-ebaaedaf9f3a None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:25.028042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66610fcb-77c5-457f-80d7-993ecc363fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:25.028787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66610fcb-77c5-457f-80d7-993ecc363fad None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:25.029139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66610fcb-77c5-457f-80d7-993ecc363fad None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:25.029691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66610fcb-77c5-457f-80d7-993ecc363fad None admin] PATH_INFO: `/v3/projects/3d132a17b1034d7f88bab67343c5a1e3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:25.030717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66610fcb-77c5-457f-80d7-993ecc363fad None admin] RBAC: Authorizing `identity:get_project(project_id=3d132a17b1034d7f88bab67343c5a1e3)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:25.032148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66610fcb-77c5-457f-80d7-993ecc363fad None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:25.034003 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 523/1054] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:54:24 2022] GET /identity/v3/projects/3d132a17b1034d7f88bab67343c5a1e3 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:25.040458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66610fcb-77c5-457f-80d7-993ecc363fad None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:25.043313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:25.060489 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1cd10-822a-4036-883e-6434b42ed6e9 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:25.096196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71bef5-d2d5-45bb-a725-55a694725a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:25.096833 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71bef5-d2d5-45bb-a725-55a694725a57 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:25.097115 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71bef5-d2d5-45bb-a725-55a694725a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:25.097387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71bef5-d2d5-45bb-a725-55a694725a57 None admin] PATH_INFO: `/v3/services/image` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:25.097969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71bef5-d2d5-45bb-a725-55a694725a57 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:25.099837 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71bef5-d2d5-45bb-a725-55a694725a57 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-ee71bef5-d2d5-45bb-a725-55a694725a57 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:25.109014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:25.110063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:25.110983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:25.110983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:25.110983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:25.110983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:25.110983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:25.110983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:25.110983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:25.110983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:25.110983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:25.110983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:25.110983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:54:25.110983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:54:25.110983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 11:54:25.110983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:25.110983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 532/1055] 10.222.0.62 () {56 vars in 1105 bytes} [Fri Sep 9 11:54:24 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:54:25.116732 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71bef5-d2d5-45bb-a725-55a694725a57 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:25.148918 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66610fcb-77c5-457f-80d7-993ecc363fad None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:25.188962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd4d6d-f3b7-4608-9ff9-f35e7e3a7d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:25.189567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd4d6d-f3b7-4608-9ff9-f35e7e3a7d59 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:25.189839 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd4d6d-f3b7-4608-9ff9-f35e7e3a7d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:25.190107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd4d6d-f3b7-4608-9ff9-f35e7e3a7d59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:25.190852 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd4d6d-f3b7-4608-9ff9-f35e7e3a7d59 None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=3d132a17b1034d7f88bab67343c5a1e3, user.id=0ac00601bdaa4f7c8b6b165f4bc47715) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:25.191155 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd4d6d-f3b7-4608-9ff9-f35e7e3a7d59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:25.192244 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:25.192621 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd4d6d-f3b7-4608-9ff9-f35e7e3a7d59 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:25.202965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 524/1056] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:54:25 2022] GET /identity/v3/role_assignments?user.id=0ac00601bdaa4f7c8b6b165f4bc47715&scope.project.id=3d132a17b1034d7f88bab67343c5a1e3&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:25.246314 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71bef5-d2d5-45bb-a725-55a694725a57 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:25.285289 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce879ed3-779f-464f-b651-ee45ab516357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:25.286517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce879ed3-779f-464f-b651-ee45ab516357 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:25.286795 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce879ed3-779f-464f-b651-ee45ab516357 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:25.287065 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce879ed3-779f-464f-b651-ee45ab516357 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:25.287721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce879ed3-779f-464f-b651-ee45ab516357 None admin] RBAC: Adding query filter params (name=image) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:25.288016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce879ed3-779f-464f-b651-ee45ab516357 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:25.289409 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce879ed3-779f-464f-b651-ee45ab516357 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:25.295399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 533/1057] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:25.304278 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd4d6d-f3b7-4608-9ff9-f35e7e3a7d59 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:25.377510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:25.416383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd4d6d-f3b7-4608-9ff9-f35e7e3a7d59 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:25.451372 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf874c6a-f481-4a15-9f10-6b6ef7651dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:25.452009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf874c6a-f481-4a15-9f10-6b6ef7651dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:25.452290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf874c6a-f481-4a15-9f10-6b6ef7651dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:25.452562 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf874c6a-f481-4a15-9f10-6b6ef7651dc1 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:25.453192 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf874c6a-f481-4a15-9f10-6b6ef7651dc1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:25.453499 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf874c6a-f481-4a15-9f10-6b6ef7651dc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:25.454895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf874c6a-f481-4a15-9f10-6b6ef7651dc1 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:25.460592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 525/1058] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:25.466440 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce879ed3-779f-464f-b651-ee45ab516357 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:25.574066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce879ed3-779f-464f-b651-ee45ab516357 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:25.608737 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376ce4c-ef48-4c07-9b88-0fce72283784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:25.609348 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376ce4c-ef48-4c07-9b88-0fce72283784 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:25.609627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376ce4c-ef48-4c07-9b88-0fce72283784 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:25.609907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376ce4c-ef48-4c07-9b88-0fce72283784 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:25.610612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376ce4c-ef48-4c07-9b88-0fce72283784 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:25.611962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376ce4c-ef48-4c07-9b88-0fce72283784 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:25.616584 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:25.634172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 534/1059] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 11:54:25 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:25.775535 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac2828-213e-4b6f-936f-d7455db88869 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:25.775747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac2828-213e-4b6f-936f-d7455db88869 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:25.775985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac2828-213e-4b6f-936f-d7455db88869 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:25.777545 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 526/1060] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:25.783112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e167fa3-0989-4fc4-b208-248572430dd0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:25.783643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e167fa3-0989-4fc4-b208-248572430dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:25.783643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e167fa3-0989-4fc4-b208-248572430dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:25.803442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5e167fa3-0989-4fc4-b208-248572430dd0 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:25.804643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5e167fa3-0989-4fc4-b208-248572430dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:25.889657 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 535/1061] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:26.154877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac2828-213e-4b6f-936f-d7455db88869 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:26.261855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac2828-213e-4b6f-936f-d7455db88869 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:26.292549 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:26.299149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf3eaa-d1ff-48bc-86cc-415b24a04dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:26.299789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf3eaa-d1ff-48bc-86cc-415b24a04dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:26.300063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf3eaa-d1ff-48bc-86cc-415b24a04dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:26.300331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf3eaa-d1ff-48bc-86cc-415b24a04dd7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:26.311576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf3eaa-d1ff-48bc-86cc-415b24a04dd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-57bf3eaa-d1ff-48bc-86cc-415b24a04dd7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:54:26.316384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:26.317901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:26.323058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:26.323058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:26.323058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:26.323058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:26.323058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:54:26.323058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:54:26.323058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:54:26.323058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:54:26.323058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:54:26.323058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:54:26.323058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:54:26.323058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:26.323058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 527/1062] 10.222.0.62 () {56 vars in 1120 bytes} [Fri Sep 9 11:54:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:26.324919 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e167fa3-0989-4fc4-b208-248572430dd0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:26.432220 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e167fa3-0989-4fc4-b208-248572430dd0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:26.449339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:26.470225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29eec87-77dd-4ed3-85df-377d2ca6ce27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:26.470846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29eec87-77dd-4ed3-85df-377d2ca6ce27 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:26.471122 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29eec87-77dd-4ed3-85df-377d2ca6ce27 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:26.471379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29eec87-77dd-4ed3-85df-377d2ca6ce27 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:26.472039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29eec87-77dd-4ed3-85df-377d2ca6ce27 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:26.472416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29eec87-77dd-4ed3-85df-377d2ca6ce27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:26.474246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29eec87-77dd-4ed3-85df-377d2ca6ce27 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:26.487523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 536/1063] 10.222.0.62 () {56 vars in 1115 bytes} [Fri Sep 9 11:54:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:26.489518 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf3eaa-d1ff-48bc-86cc-415b24a04dd7 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:26.598508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf3eaa-d1ff-48bc-86cc-415b24a04dd7 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:26.634280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b2d63-b431-4590-b09e-5849ca6da9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:26.634909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b2d63-b431-4590-b09e-5849ca6da9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:26.635189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b2d63-b431-4590-b09e-5849ca6da9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:26.635455 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b2d63-b431-4590-b09e-5849ca6da9d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:26.646344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b2d63-b431-4590-b09e-5849ca6da9d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:26.647793 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b2d63-b431-4590-b09e-5849ca6da9d0 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-4e5b2d63-b431-4590-b09e-5849ca6da9d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:54:26.663380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:26.664516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:54:26.665419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:54:26.665419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:26.665419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:26.665419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 528/1064] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:26.671466 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29eec87-77dd-4ed3-85df-377d2ca6ce27 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:26.776248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29eec87-77dd-4ed3-85df-377d2ca6ce27 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:26.811475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a302796-8f11-455a-a073-a9e7645cb2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:26.812198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a302796-8f11-455a-a073-a9e7645cb2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:26.812345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a302796-8f11-455a-a073-a9e7645cb2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:26.812607 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a302796-8f11-455a-a073-a9e7645cb2b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:26.813254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a302796-8f11-455a-a073-a9e7645cb2b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:26.813551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a302796-8f11-455a-a073-a9e7645cb2b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:26.814935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a302796-8f11-455a-a073-a9e7645cb2b7 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:26.829112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 537/1065] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:26.833905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b2d63-b431-4590-b09e-5849ca6da9d0 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:26.881597 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:26.952332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b2d63-b431-4590-b09e-5849ca6da9d0 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:26.990768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988724e-d746-4da2-a327-b82ea024632e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:26.991413 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988724e-d746-4da2-a327-b82ea024632e None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:26.991689 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988724e-d746-4da2-a327-b82ea024632e None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:26.991961 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988724e-d746-4da2-a327-b82ea024632e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:27.000589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988724e-d746-4da2-a327-b82ea024632e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:27.002067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988724e-d746-4da2-a327-b82ea024632e None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-5988724e-d746-4da2-a327-b82ea024632e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:54:27.015251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:27.016322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:27.017398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:27.017398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:27.017398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:54:27.017398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:54:27.017398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:54:27.017398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:54:27.017398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:54:27.017398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:54:27.017398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:54:27.017398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:27.017398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 529/1066] 10.222.0.62 () {56 vars in 1093 bytes} [Fri Sep 9 11:54:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:27.024385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a302796-8f11-455a-a073-a9e7645cb2b7 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:27.080059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:27.146557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a302796-8f11-455a-a073-a9e7645cb2b7 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:27.182476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531fd91-0440-4aa8-ae0f-bccf8d2080a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:27.183088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531fd91-0440-4aa8-ae0f-bccf8d2080a9 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:27.183364 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531fd91-0440-4aa8-ae0f-bccf8d2080a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:27.183628 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531fd91-0440-4aa8-ae0f-bccf8d2080a9 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:27.184346 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531fd91-0440-4aa8-ae0f-bccf8d2080a9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:27.184646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531fd91-0440-4aa8-ae0f-bccf8d2080a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:27.186044 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531fd91-0440-4aa8-ae0f-bccf8d2080a9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:27.206473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 538/1067] 10.222.0.62 () {56 vars in 1133 bytes} [Fri Sep 9 11:54:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:27.212317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988724e-d746-4da2-a327-b82ea024632e None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:27.338640 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988724e-d746-4da2-a327-b82ea024632e None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:27.340578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:27.377972 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24dfe5-9444-4711-9a9d-25e6b7428ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:27.378646 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24dfe5-9444-4711-9a9d-25e6b7428ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:27.378932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24dfe5-9444-4711-9a9d-25e6b7428ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:27.379193 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24dfe5-9444-4711-9a9d-25e6b7428ab7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:27.392163 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24dfe5-9444-4711-9a9d-25e6b7428ab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:27.393611 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24dfe5-9444-4711-9a9d-25e6b7428ab7 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-2b24dfe5-9444-4711-9a9d-25e6b7428ab7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:54:27.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:27.410881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:54:27.411865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:54:27.411865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:27.411865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:27.411865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 530/1068] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:27.417889 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531fd91-0440-4aa8-ae0f-bccf8d2080a9 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:27.424761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e21b21-c5b9-4b11-9e83-fd34356c7ebb None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:27.425078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e21b21-c5b9-4b11-9e83-fd34356c7ebb None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:27.425305 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e21b21-c5b9-4b11-9e83-fd34356c7ebb None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:27.427567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 531/1069] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:54:27.432720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4392e2-5e75-4ac6-8ec5-926b765c4d4e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:27.432995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4392e2-5e75-4ac6-8ec5-926b765c4d4e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:27.433262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4392e2-5e75-4ac6-8ec5-926b765c4d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:27.452869 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-0c4392e2-5e75-4ac6-8ec5-926b765c4d4e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:27.454134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0c4392e2-5e75-4ac6-8ec5-926b765c4d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:27.527574 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 532/1070] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:54:27.538976 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:27.542860 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531fd91-0440-4aa8-ae0f-bccf8d2080a9 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:27.579546 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f1056-5755-4185-9437-8f779614fdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:27.580156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f1056-5755-4185-9437-8f779614fdab None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:27.580428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f1056-5755-4185-9437-8f779614fdab None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:27.580693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f1056-5755-4185-9437-8f779614fdab None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:27.581333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f1056-5755-4185-9437-8f779614fdab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:27.581641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f1056-5755-4185-9437-8f779614fdab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:27.583577 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f1056-5755-4185-9437-8f779614fdab None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:27.595883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 539/1071] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:27.601885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4392e2-5e75-4ac6-8ec5-926b765c4d4e None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:27.694071 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:27.713318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4392e2-5e75-4ac6-8ec5-926b765c4d4e None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:27.754495 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5a1d2-dca0-4c15-a29e-a847b9361697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:27.755164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5a1d2-dca0-4c15-a29e-a847b9361697 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:27.755442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5a1d2-dca0-4c15-a29e-a847b9361697 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:27.755710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5a1d2-dca0-4c15-a29e-a847b9361697 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:27.763955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5a1d2-dca0-4c15-a29e-a847b9361697 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:27.765441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5a1d2-dca0-4c15-a29e-a847b9361697 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:27.778941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-49f5a1d2-dca0-4c15-a29e-a847b9361697 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:54:27.778941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:27.778941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:27.778941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:27.778941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:27.778941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:27.778941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:27.778941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:27.778941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:27.778941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:27.778941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:27.778941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:27.778941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:54:27.778941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:54:27.778941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:27.780048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:27.781468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:54:27.781468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:54:27.781468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:54:27.781468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:54:27.781468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:54:27.781468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:54:27.781468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:54:27.781468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:27.782474 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 533/1072] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:54:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:27.786515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f1056-5755-4185-9437-8f779614fdab None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:27.805701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5a1d2-dca0-4c15-a29e-a847b9361697 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:27.891098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f1056-5755-4185-9437-8f779614fdab None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:27.925792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0de3d3-70a9-4f22-845b-c6b6e639d870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:27.926507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0de3d3-70a9-4f22-845b-c6b6e639d870 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:27.926787 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0de3d3-70a9-4f22-845b-c6b6e639d870 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:27.927060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0de3d3-70a9-4f22-845b-c6b6e639d870 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:27.927753 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0de3d3-70a9-4f22-845b-c6b6e639d870 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:27.928059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0de3d3-70a9-4f22-845b-c6b6e639d870 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:27.928411 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5a1d2-dca0-4c15-a29e-a847b9361697 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:27.929487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0de3d3-70a9-4f22-845b-c6b6e639d870 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:27.944734 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 540/1073] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:54:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:27.946083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07654de9-08ef-4f92-adb8-330921a3a173 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:27.946399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07654de9-08ef-4f92-adb8-330921a3a173 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:27.946670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07654de9-08ef-4f92-adb8-330921a3a173 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:27.948368 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 541/1074] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:27.951024 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07654de9-08ef-4f92-adb8-330921a3a173 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:27.969623 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:27.971774 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d56796-1358-455a-86e5-7d39ddec63cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:27.972687 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d56796-1358-455a-86e5-7d39ddec63cd None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:27.972964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d56796-1358-455a-86e5-7d39ddec63cd None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:27.973231 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d56796-1358-455a-86e5-7d39ddec63cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:27.984687 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d56796-1358-455a-86e5-7d39ddec63cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-a7d56796-1358-455a-86e5-7d39ddec63cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:54:27.989538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:27.995900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:27.996811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:27.996811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:27.996811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:27.996811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:27.996811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:54:27.996811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:54:27.996811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:54:27.996811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:54:27.996811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:54:27.996811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:54:27.996811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:54:27.996811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:27.996811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 534/1075] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:54:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:27.996811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1173f-820c-4e6a-9945-71437d297bb3 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:27.997616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1173f-820c-4e6a-9945-71437d297bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:27.997616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1173f-820c-4e6a-9945-71437d297bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:27.999765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:28.012425 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-eeb1173f-820c-4e6a-9945-71437d297bb3 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:28.013631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-eeb1173f-820c-4e6a-9945-71437d297bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:28.062869 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07654de9-08ef-4f92-adb8-330921a3a173 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:28.087224 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 535/1076] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:28.088331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1173f-820c-4e6a-9945-71437d297bb3 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:28.097960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e4b8e-3761-4f41-9b24-7f7f1276996c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:28.098665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e4b8e-3761-4f41-9b24-7f7f1276996c None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:28.098980 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e4b8e-3761-4f41-9b24-7f7f1276996c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:28.099252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e4b8e-3761-4f41-9b24-7f7f1276996c None admin] PATH_INFO: `/v3/projects/32b06df0c5504592b012585f193386e7/users/1b30e80c421245b5a1ed5849b28ea90e/roles/335f4cdc5c87464a8f0aedd8b9831e61` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:28.101507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e4b8e-3761-4f41-9b24-7f7f1276996c None admin] RBAC: Authorizing `identity:create_grant(project_id=32b06df0c5504592b012585f193386e7, user_id=1b30e80c421245b5a1ed5849b28ea90e, role_id=335f4cdc5c87464a8f0aedd8b9831e61)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:28.102968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e4b8e-3761-4f41-9b24-7f7f1276996c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:28.119261 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 542/1077] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:54:27 2022] PUT /identity/v3/projects/32b06df0c5504592b012585f193386e7/users/1b30e80c421245b5a1ed5849b28ea90e/roles/335f4cdc5c87464a8f0aedd8b9831e61 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:54:28.236321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1173f-820c-4e6a-9945-71437d297bb3 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:28.282427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5677e-c9fc-4e26-9315-9a9d52045232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:28.283476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5677e-c9fc-4e26-9315-9a9d52045232 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:28.283991 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5677e-c9fc-4e26-9315-9a9d52045232 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:28.284485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5677e-c9fc-4e26-9315-9a9d52045232 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:28.285411 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5677e-c9fc-4e26-9315-9a9d52045232 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:28.285991 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5677e-c9fc-4e26-9315-9a9d52045232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:28.287660 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5677e-c9fc-4e26-9315-9a9d52045232 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:28.289590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:28.302588 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 536/1078] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:54:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:28.308889 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e4b8e-3761-4f41-9b24-7f7f1276996c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:28.361230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5677e-c9fc-4e26-9315-9a9d52045232 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:28.408345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:28.435642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e4b8e-3761-4f41-9b24-7f7f1276996c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:28.473314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5677e-c9fc-4e26-9315-9a9d52045232 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:28.475747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9b7f2-b8ec-4169-84fe-b6ce4236ace4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:28.476324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9b7f2-b8ec-4169-84fe-b6ce4236ace4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:28.476605 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9b7f2-b8ec-4169-84fe-b6ce4236ace4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:28.476881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9b7f2-b8ec-4169-84fe-b6ce4236ace4 None admin] PATH_INFO: `/v3/users/0ac00601bdaa4f7c8b6b165f4bc47715` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:28.478062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9b7f2-b8ec-4169-84fe-b6ce4236ace4 None admin] RBAC: Authorizing `identity:get_user(user_id=0ac00601bdaa4f7c8b6b165f4bc47715)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:28.479512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9b7f2-b8ec-4169-84fe-b6ce4236ace4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:28.481279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 543/1079] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:54:28 2022] GET /identity/v3/users/0ac00601bdaa4f7c8b6b165f4bc47715 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:28.486958 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9b7f2-b8ec-4169-84fe-b6ce4236ace4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:28.510834 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38959d42-0563-4e5c-bec0-991a23c8f042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:28.511541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38959d42-0563-4e5c-bec0-991a23c8f042 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:28.511747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38959d42-0563-4e5c-bec0-991a23c8f042 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:28.511989 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38959d42-0563-4e5c-bec0-991a23c8f042 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:28.586776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:28.596085 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38959d42-0563-4e5c-bec0-991a23c8f042 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:28.597590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38959d42-0563-4e5c-bec0-991a23c8f042 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:28.606440 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9b7f2-b8ec-4169-84fe-b6ce4236ace4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:28.646785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504444b-520d-4544-af47-ed776e1ea781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:28.647747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504444b-520d-4544-af47-ed776e1ea781 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:28.647945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504444b-520d-4544-af47-ed776e1ea781 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:28.648222 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504444b-520d-4544-af47-ed776e1ea781 None admin] PATH_INFO: `/v3/projects/3d132a17b1034d7f88bab67343c5a1e3` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:28.649550 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504444b-520d-4544-af47-ed776e1ea781 None admin] RBAC: Authorizing `identity:get_project(project_id=3d132a17b1034d7f88bab67343c5a1e3)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:28.650998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504444b-520d-4544-af47-ed776e1ea781 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:28.652814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 544/1080] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:54:28 2022] GET /identity/v3/projects/3d132a17b1034d7f88bab67343c5a1e3 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:28.658605 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504444b-520d-4544-af47-ed776e1ea781 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:28.678712 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:28.707335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 537/1081] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:54:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:54:28.713618 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38959d42-0563-4e5c-bec0-991a23c8f042 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:28.782536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504444b-520d-4544-af47-ed776e1ea781 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:28.819874 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:28.827163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233e213-f425-43bf-8a55-db39847a7232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:28.827360 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38959d42-0563-4e5c-bec0-991a23c8f042 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:28.827846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233e213-f425-43bf-8a55-db39847a7232 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:28.828113 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233e213-f425-43bf-8a55-db39847a7232 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:28.828389 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233e213-f425-43bf-8a55-db39847a7232 None admin] PATH_INFO: `/v3/projects/3d132a17b1034d7f88bab67343c5a1e3/users/0ac00601bdaa4f7c8b6b165f4bc47715/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:28.830394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233e213-f425-43bf-8a55-db39847a7232 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d132a17b1034d7f88bab67343c5a1e3, user_id=0ac00601bdaa4f7c8b6b165f4bc47715, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:28.831827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233e213-f425-43bf-8a55-db39847a7232 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:28.858163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 545/1082] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:54:28 2022] PUT /identity/v3/projects/3d132a17b1034d7f88bab67343c5a1e3/users/0ac00601bdaa4f7c8b6b165f4bc47715/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:54:28.872784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67dab0-0add-4ee1-a646-021fcde4862c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:28.873542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67dab0-0add-4ee1-a646-021fcde4862c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:28.873889 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67dab0-0add-4ee1-a646-021fcde4862c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:28.874266 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67dab0-0add-4ee1-a646-021fcde4862c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:28.888711 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67dab0-0add-4ee1-a646-021fcde4862c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:28.890335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67dab0-0add-4ee1-a646-021fcde4862c None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-7f67dab0-0add-4ee1-a646-021fcde4862c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:54:28.907581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:28.908669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:54:28.910872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:54:28.910872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:28.910872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:28.910872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 538/1083] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:54:28.915099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233e213-f425-43bf-8a55-db39847a7232 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:29.001024 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:29.041816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233e213-f425-43bf-8a55-db39847a7232 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:29.079085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d768e85-6022-4e76-b895-4533593b757b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:29.079703 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d768e85-6022-4e76-b895-4533593b757b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:29.079997 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d768e85-6022-4e76-b895-4533593b757b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:29.080321 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d768e85-6022-4e76-b895-4533593b757b None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:29.080973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d768e85-6022-4e76-b895-4533593b757b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:29.081299 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d768e85-6022-4e76-b895-4533593b757b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:29.083396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d768e85-6022-4e76-b895-4533593b757b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:29.096248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 546/1084] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:30.339884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867508c2-aa2b-4345-91db-13aa126ce362 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:30.340417 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867508c2-aa2b-4345-91db-13aa126ce362 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:30.340417 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867508c2-aa2b-4345-91db-13aa126ce362 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:30.342258 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 539/1085] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:30.347140 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db27ca-091a-45bc-bdc5-1b3d2cf69679 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:30.347578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db27ca-091a-45bc-bdc5-1b3d2cf69679 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:30.347699 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db27ca-091a-45bc-bdc5-1b3d2cf69679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:30.366982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-19db27ca-091a-45bc-bdc5-1b3d2cf69679 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:30.368166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-19db27ca-091a-45bc-bdc5-1b3d2cf69679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:30.436384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 547/1086] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:30.709454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867508c2-aa2b-4345-91db-13aa126ce362 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:30.812325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867508c2-aa2b-4345-91db-13aa126ce362 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:30.844641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ca60f-d7fc-4acc-bc83-24e367589f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:30.845269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ca60f-d7fc-4acc-bc83-24e367589f73 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:30.845543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ca60f-d7fc-4acc-bc83-24e367589f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:30.845810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ca60f-d7fc-4acc-bc83-24e367589f73 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:30.856864 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ca60f-d7fc-4acc-bc83-24e367589f73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-242ca60f-d7fc-4acc-bc83-24e367589f73 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:54:30.861996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:30.863178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:30.864933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:30.864933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:30.864933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:30.864933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:30.864933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:54:30.864933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:54:30.864933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:54:30.864933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:54:30.864933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:54:30.864933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:54:30.864933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 11:54:30.864933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:30.864933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 540/1087] 10.222.0.62 () {56 vars in 1120 bytes} [Fri Sep 9 11:54:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:30.870062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db27ca-091a-45bc-bdc5-1b3d2cf69679 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:30.932343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:30.974200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db27ca-091a-45bc-bdc5-1b3d2cf69679 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:31.010985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1cbf3-7e9e-4a50-a8f5-7b3fc10b121f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:31.011843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1cbf3-7e9e-4a50-a8f5-7b3fc10b121f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:31.012164 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1cbf3-7e9e-4a50-a8f5-7b3fc10b121f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:31.012439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1cbf3-7e9e-4a50-a8f5-7b3fc10b121f None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:31.013113 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1cbf3-7e9e-4a50-a8f5-7b3fc10b121f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:31.013415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1cbf3-7e9e-4a50-a8f5-7b3fc10b121f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:31.014810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1cbf3-7e9e-4a50-a8f5-7b3fc10b121f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:31.023311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 548/1088] 10.222.0.62 () {56 vars in 1115 bytes} [Fri Sep 9 11:54:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:31.029160 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ca60f-d7fc-4acc-bc83-24e367589f73 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:31.041130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:31.044680 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67e762-915d-4446-bc82-2f8f88265bd6 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:31.044968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67e762-915d-4446-bc82-2f8f88265bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:31.045244 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67e762-915d-4446-bc82-2f8f88265bd6 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:31.046633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 549/1089] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:31.052358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb129703-a839-4aeb-95a8-00ee55c4afe7 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:31.052639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb129703-a839-4aeb-95a8-00ee55c4afe7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:31.052910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb129703-a839-4aeb-95a8-00ee55c4afe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:31.070550 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-bb129703-a839-4aeb-95a8-00ee55c4afe7 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:31.071730 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bb129703-a839-4aeb-95a8-00ee55c4afe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:31.132497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ca60f-d7fc-4acc-bc83-24e367589f73 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:31.139155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 550/1090] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:54:31.166710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79926c9e-18c2-4118-ac79-2b9883a9892d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:31.167342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79926c9e-18c2-4118-ac79-2b9883a9892d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:31.167619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79926c9e-18c2-4118-ac79-2b9883a9892d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:31.167888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79926c9e-18c2-4118-ac79-2b9883a9892d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:31.179083 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79926c9e-18c2-4118-ac79-2b9883a9892d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:31.180533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79926c9e-18c2-4118-ac79-2b9883a9892d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-79926c9e-18c2-4118-ac79-2b9883a9892d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:54:31.194558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:31.195525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:54:31.196747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:54:31.196747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:31.196747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:31.196747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 541/1091] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:54:31.201320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb129703-a839-4aeb-95a8-00ee55c4afe7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:31.258925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:31.303123 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb129703-a839-4aeb-95a8-00ee55c4afe7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:31.337347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51279222-3a40-437d-b641-33d9f47a0399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:31.338006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51279222-3a40-437d-b641-33d9f47a0399 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:31.338312 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51279222-3a40-437d-b641-33d9f47a0399 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:31.338583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51279222-3a40-437d-b641-33d9f47a0399 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:31.339213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51279222-3a40-437d-b641-33d9f47a0399 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:31.339512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51279222-3a40-437d-b641-33d9f47a0399 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:31.340920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51279222-3a40-437d-b641-33d9f47a0399 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:31.352733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 551/1092] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:31.358786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79926c9e-18c2-4118-ac79-2b9883a9892d None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:31.413435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51279222-3a40-437d-b641-33d9f47a0399 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:31.417563 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:31.464831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79926c9e-18c2-4118-ac79-2b9883a9892d None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:31.499724 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4e409-f9c7-4091-8428-e01a57eb9799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:31.500271 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4e409-f9c7-4091-8428-e01a57eb9799 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:31.500555 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4e409-f9c7-4091-8428-e01a57eb9799 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:31.500823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4e409-f9c7-4091-8428-e01a57eb9799 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:31.508451 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4e409-f9c7-4091-8428-e01a57eb9799 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:31.509876 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4e409-f9c7-4091-8428-e01a57eb9799 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:31.516788 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51279222-3a40-437d-b641-33d9f47a0399 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-3ec4e409-f9c7-4091-8428-e01a57eb9799 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:54:31.521638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:31.522629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:31.523570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:31.523570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:31.523570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:54:31.523570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:54:31.523570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:54:31.523570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:54:31.523570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:54:31.523570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:54:31.523570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:54:31.523570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:31.524103 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 542/1093] 10.222.0.62 () {56 vars in 1093 bytes} [Fri Sep 9 11:54:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:31.528590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4e409-f9c7-4091-8428-e01a57eb9799 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:31.531509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:31.554401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b121a5c-b9aa-4375-ab17-1962b74c45d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:31.555032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b121a5c-b9aa-4375-ab17-1962b74c45d0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:31.555304 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b121a5c-b9aa-4375-ab17-1962b74c45d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:31.555572 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b121a5c-b9aa-4375-ab17-1962b74c45d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:31.565816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b121a5c-b9aa-4375-ab17-1962b74c45d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-7b121a5c-b9aa-4375-ab17-1962b74c45d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:54:31.571185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:31.572174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:31.573232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:31.573232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:31.573232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:31.573232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:31.573232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:54:31.573232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:54:31.573232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:54:31.573232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:54:31.573232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:54:31.573232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:54:31.573232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:54:31.573232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:31.573232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 552/1094] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:54:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:54:31.578344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b121a5c-b9aa-4375-ab17-1962b74c45d0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:31.630557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4e409-f9c7-4091-8428-e01a57eb9799 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:31.650968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:31.665510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e83d8-e560-49b5-a0ff-ddc16d247c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:31.666205 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e83d8-e560-49b5-a0ff-ddc16d247c77 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:31.666428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e83d8-e560-49b5-a0ff-ddc16d247c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:31.666688 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e83d8-e560-49b5-a0ff-ddc16d247c77 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:31.667417 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e83d8-e560-49b5-a0ff-ddc16d247c77 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:31.667725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e83d8-e560-49b5-a0ff-ddc16d247c77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:31.669156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e83d8-e560-49b5-a0ff-ddc16d247c77 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:31.680752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b121a5c-b9aa-4375-ab17-1962b74c45d0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:31.687341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 543/1095] 10.222.0.62 () {56 vars in 1133 bytes} [Fri Sep 9 11:54:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:31.693220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e83d8-e560-49b5-a0ff-ddc16d247c77 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:31.713557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de3b97-4d4c-47b2-bd44-06c9efc631b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:31.714183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de3b97-4d4c-47b2-bd44-06c9efc631b5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:31.714461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de3b97-4d4c-47b2-bd44-06c9efc631b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:31.714725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de3b97-4d4c-47b2-bd44-06c9efc631b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:31.715374 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de3b97-4d4c-47b2-bd44-06c9efc631b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:31.715672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de3b97-4d4c-47b2-bd44-06c9efc631b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:31.717012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de3b97-4d4c-47b2-bd44-06c9efc631b5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:31.725518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 553/1096] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:54:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:31.731359 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de3b97-4d4c-47b2-bd44-06c9efc631b5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:31.767222 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:31.793639 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e83d8-e560-49b5-a0ff-ddc16d247c77 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:31.828944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df660b8c-69db-484e-a388-17834ed1148a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:31.829544 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df660b8c-69db-484e-a388-17834ed1148a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:31.829820 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df660b8c-69db-484e-a388-17834ed1148a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:31.830089 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df660b8c-69db-484e-a388-17834ed1148a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:31.830898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de3b97-4d4c-47b2-bd44-06c9efc631b5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:31.867148 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211558f8-864d-4792-8edd-a16997e3789d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:31.867802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211558f8-864d-4792-8edd-a16997e3789d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:31.868058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211558f8-864d-4792-8edd-a16997e3789d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:31.868326 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211558f8-864d-4792-8edd-a16997e3789d None admin] PATH_INFO: `/v3/users/0ac00601bdaa4f7c8b6b165f4bc47715` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:31.869458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211558f8-864d-4792-8edd-a16997e3789d None admin] RBAC: Authorizing `identity:get_user(user_id=0ac00601bdaa4f7c8b6b165f4bc47715)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:31.870936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211558f8-864d-4792-8edd-a16997e3789d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:31.872524 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 554/1097] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:54:31 2022] GET /identity/v3/users/0ac00601bdaa4f7c8b6b165f4bc47715 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:31.878095 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211558f8-864d-4792-8edd-a16997e3789d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:31.881349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:31.904734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df660b8c-69db-484e-a388-17834ed1148a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:31.906203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df660b8c-69db-484e-a388-17834ed1148a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:31.981072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211558f8-864d-4792-8edd-a16997e3789d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:31.983962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:32.008853 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 544/1098] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:54:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 11:54:32.014847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df660b8c-69db-484e-a388-17834ed1148a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:32.022704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325b978-d680-41af-8fea-b673db446d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:32.023290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325b978-d680-41af-8fea-b673db446d1a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:32.023568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325b978-d680-41af-8fea-b673db446d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:32.023834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325b978-d680-41af-8fea-b673db446d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:32.090551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325b978-d680-41af-8fea-b673db446d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:32.091866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:32.091991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325b978-d680-41af-8fea-b673db446d1a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:32.118179 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df660b8c-69db-484e-a388-17834ed1148a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:32.151583 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c60408-5c7e-4a81-9e59-295a49315bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:32.152277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c60408-5c7e-4a81-9e59-295a49315bbd None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:32.152473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c60408-5c7e-4a81-9e59-295a49315bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:32.152737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c60408-5c7e-4a81-9e59-295a49315bbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:32.163925 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c60408-5c7e-4a81-9e59-295a49315bbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:32.165377 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c60408-5c7e-4a81-9e59-295a49315bbd None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-01c60408-5c7e-4a81-9e59-295a49315bbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:54:32.180037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:32.181027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:54:32.181977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:54:32.181977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:54:32.181977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:32.181977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 545/1099] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:32.186807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c60408-5c7e-4a81-9e59-295a49315bbd None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:32.198382 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 555/1100] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:54:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 11:54:32.204168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325b978-d680-41af-8fea-b673db446d1a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:32.207097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:32.287537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c60408-5c7e-4a81-9e59-295a49315bbd None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:32.303882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325b978-d680-41af-8fea-b673db446d1a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:32.315628 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:32.322617 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f89653-c238-48ea-881d-8978f829563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:32.323227 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f89653-c238-48ea-881d-8978f829563e None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:32.323510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f89653-c238-48ea-881d-8978f829563e None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:32.323784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f89653-c238-48ea-881d-8978f829563e None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:32.324427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f89653-c238-48ea-881d-8978f829563e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:32.324737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f89653-c238-48ea-881d-8978f829563e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:32.326163 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f89653-c238-48ea-881d-8978f829563e None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:32.338678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e9019-e0f2-42e2-b5ab-17068f2dc0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:32.339244 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 546/1101] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:32.342419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e9019-e0f2-42e2-b5ab-17068f2dc0ce None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:32.342801 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e9019-e0f2-42e2-b5ab-17068f2dc0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:32.343172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e9019-e0f2-42e2-b5ab-17068f2dc0ce None admin] PATH_INFO: `/v3/projects/3d132a17b1034d7f88bab67343c5a1e3` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:32.345211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e9019-e0f2-42e2-b5ab-17068f2dc0ce None admin] RBAC: Authorizing `identity:get_project(project_id=3d132a17b1034d7f88bab67343c5a1e3)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:32.345873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f89653-c238-48ea-881d-8978f829563e None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:32.347764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e9019-e0f2-42e2-b5ab-17068f2dc0ce None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:32.351009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 556/1102] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:54:32 2022] GET /identity/v3/projects/3d132a17b1034d7f88bab67343c5a1e3 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:32.357424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e9019-e0f2-42e2-b5ab-17068f2dc0ce None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:32.425434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:32.450496 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f89653-c238-48ea-881d-8978f829563e None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:32.455986 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e9019-e0f2-42e2-b5ab-17068f2dc0ce None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:32.484611 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8555d-ba75-4396-b3d8-dd311b706d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:32.485251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8555d-ba75-4396-b3d8-dd311b706d3b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:32.485528 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8555d-ba75-4396-b3d8-dd311b706d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:32.485795 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8555d-ba75-4396-b3d8-dd311b706d3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:32.491860 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d10ea-0379-4cff-9233-0b768e1d9566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:32.492466 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d10ea-0379-4cff-9233-0b768e1d9566 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:32.492745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d10ea-0379-4cff-9233-0b768e1d9566 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:32.492880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8555d-ba75-4396-b3d8-dd311b706d3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:32.493020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d10ea-0379-4cff-9233-0b768e1d9566 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:32.493752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d10ea-0379-4cff-9233-0b768e1d9566 None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=3d132a17b1034d7f88bab67343c5a1e3, user.id=0ac00601bdaa4f7c8b6b165f4bc47715) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:32.494054 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d10ea-0379-4cff-9233-0b768e1d9566 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:32.494358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8555d-ba75-4396-b3d8-dd311b706d3b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:32.495491 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d10ea-0379-4cff-9233-0b768e1d9566 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:32.505253 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 557/1103] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:54:32 2022] GET /identity/v3/role_assignments?user.id=0ac00601bdaa4f7c8b6b165f4bc47715&scope.project.id=3d132a17b1034d7f88bab67343c5a1e3&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:32.507752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-2da8555d-ba75-4396-b3d8-dd311b706d3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:54:32.507752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:32.507752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:32.507752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:32.507752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:32.507752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:32.507752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:32.507752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:32.507752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:32.507752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:32.507752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:32.507752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:32.507752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:54:32.507752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:54:32.507752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:32.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:32.510749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:54:32.510749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:54:32.510749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:54:32.510749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:54:32.510749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:54:32.510749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:54:32.510749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:54:32.510749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:54:32.510749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 547/1104] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:54:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:54:32.515252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d10ea-0379-4cff-9233-0b768e1d9566 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:32.624210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d10ea-0379-4cff-9233-0b768e1d9566 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:32.661232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67faf0-9975-4500-86aa-c92a6a554f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:32.661852 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67faf0-9975-4500-86aa-c92a6a554f52 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:32.662146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67faf0-9975-4500-86aa-c92a6a554f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:32.662423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67faf0-9975-4500-86aa-c92a6a554f52 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:32.663116 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67faf0-9975-4500-86aa-c92a6a554f52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:32.663420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67faf0-9975-4500-86aa-c92a6a554f52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:32.664847 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67faf0-9975-4500-86aa-c92a6a554f52 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:32.668189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:32.679663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 558/1105] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:54:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:32.685832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8555d-ba75-4396-b3d8-dd311b706d3b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:32.739820 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:32.789403 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8555d-ba75-4396-b3d8-dd311b706d3b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:32.823519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53453c0b-22fe-456f-9375-01fc79909e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:32.824115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53453c0b-22fe-456f-9375-01fc79909e71 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:32.824388 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53453c0b-22fe-456f-9375-01fc79909e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:32.824651 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53453c0b-22fe-456f-9375-01fc79909e71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:32.825366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53453c0b-22fe-456f-9375-01fc79909e71 None admin] RBAC: Adding query filter params (role.id=335f4cdc5c87464a8f0aedd8b9831e61, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=1b30e80c421245b5a1ed5849b28ea90e) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:32.825665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53453c0b-22fe-456f-9375-01fc79909e71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:32.827098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53453c0b-22fe-456f-9375-01fc79909e71 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:32.832773 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 548/1106] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:54:32 2022] GET /identity/v3/role_assignments?user.id=1b30e80c421245b5a1ed5849b28ea90e&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=335f4cdc5c87464a8f0aedd8b9831e61 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:34.707395 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0081f-b52c-498f-bf07-3dc772c0fbbb None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:34.707933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0081f-b52c-498f-bf07-3dc772c0fbbb None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:34.707933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0081f-b52c-498f-bf07-3dc772c0fbbb None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:34.709295 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 559/1107] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:34.714551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b0150-0066-4b12-ac43-a97d82ff5219 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:34.714953 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b0150-0066-4b12-ac43-a97d82ff5219 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:34.715100 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b0150-0066-4b12-ac43-a97d82ff5219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:34.733573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-3b4b0150-0066-4b12-ac43-a97d82ff5219 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:34.734782 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-3b4b0150-0066-4b12-ac43-a97d82ff5219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:34.798979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 549/1108] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:35.064192 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0081f-b52c-498f-bf07-3dc772c0fbbb None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:35.164093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0081f-b52c-498f-bf07-3dc772c0fbbb None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:35.197823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b216d3a-02ca-4a03-af20-9e3938c08565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:35.198479 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b216d3a-02ca-4a03-af20-9e3938c08565 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:35.198756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b216d3a-02ca-4a03-af20-9e3938c08565 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:35.199025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b216d3a-02ca-4a03-af20-9e3938c08565 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:35.200138 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b216d3a-02ca-4a03-af20-9e3938c08565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:35.201550 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b216d3a-02ca-4a03-af20-9e3938c08565 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:35.203164 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 560/1109] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:35.209126 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b0150-0066-4b12-ac43-a97d82ff5219 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:35.254867 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:35.308767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b0150-0066-4b12-ac43-a97d82ff5219 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:35.341185 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306a450-76ab-4789-8a4b-1a751dd973cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:35.341791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306a450-76ab-4789-8a4b-1a751dd973cb None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:35.342070 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306a450-76ab-4789-8a4b-1a751dd973cb None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:35.342368 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306a450-76ab-4789-8a4b-1a751dd973cb None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:35.343008 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306a450-76ab-4789-8a4b-1a751dd973cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:35.344453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306a450-76ab-4789-8a4b-1a751dd973cb None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:35.366215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 550/1110] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:54:35 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:37.554312 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5d71e-63f6-4f84-93ac-4a497867afba None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:37.554814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5d71e-63f6-4f84-93ac-4a497867afba None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:37.554814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5d71e-63f6-4f84-93ac-4a497867afba None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:37.556115 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 561/1111] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:37.561649 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe5c48-bf24-4cc0-9b8b-5a20124788db None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:37.562088 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe5c48-bf24-4cc0-9b8b-5a20124788db None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:37.562210 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe5c48-bf24-4cc0-9b8b-5a20124788db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:37.580084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d6fe5c48-bf24-4cc0-9b8b-5a20124788db None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:37.581269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d6fe5c48-bf24-4cc0-9b8b-5a20124788db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:37.646779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 551/1112] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:37.933230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5d71e-63f6-4f84-93ac-4a497867afba None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:37.998025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:38.036820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5d71e-63f6-4f84-93ac-4a497867afba None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:38.072123 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fece5-964b-40cc-ab4d-87a69d3d6acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:38.072746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fece5-964b-40cc-ab4d-87a69d3d6acb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:38.073024 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fece5-964b-40cc-ab4d-87a69d3d6acb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:38.073294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fece5-964b-40cc-ab4d-87a69d3d6acb None admin] PATH_INFO: `/v3/roles/d43148a29472443ba7dfe2424ec60baf` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:38.074700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fece5-964b-40cc-ab4d-87a69d3d6acb None admin] RBAC: Authorizing `identity:get_role(role_id=d43148a29472443ba7dfe2424ec60baf)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:38.076052 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fece5-964b-40cc-ab4d-87a69d3d6acb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:38.077387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 562/1113] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:54:37 2022] GET /identity/v3/roles/d43148a29472443ba7dfe2424ec60baf => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:38.083826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe5c48-bf24-4cc0-9b8b-5a20124788db None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:38.113262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:38.185105 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe5c48-bf24-4cc0-9b8b-5a20124788db None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:38.218370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01469a1c-79a2-4fc2-b4b6-26bb3ba23dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:38.218994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01469a1c-79a2-4fc2-b4b6-26bb3ba23dce None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:38.219272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01469a1c-79a2-4fc2-b4b6-26bb3ba23dce None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:38.219540 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01469a1c-79a2-4fc2-b4b6-26bb3ba23dce None admin] PATH_INFO: `/v3/users/7c2b6fe1904a4bdc93711b1a75a6602a` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:38.233764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01469a1c-79a2-4fc2-b4b6-26bb3ba23dce None admin] RBAC: Authorizing `identity:get_user(user_id=7c2b6fe1904a4bdc93711b1a75a6602a)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:38.235269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01469a1c-79a2-4fc2-b4b6-26bb3ba23dce None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:38.236887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 552/1114] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:54:38 2022] GET /identity/v3/users/7c2b6fe1904a4bdc93711b1a75a6602a => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:38.242699 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fece5-964b-40cc-ab4d-87a69d3d6acb None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:38.342549 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fece5-964b-40cc-ab4d-87a69d3d6acb None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:38.370711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:38.375848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5b652-ff3d-4b9b-b2c9-c21f107a8227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:38.376458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5b652-ff3d-4b9b-b2c9-c21f107a8227 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:38.376731 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5b652-ff3d-4b9b-b2c9-c21f107a8227 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:38.376998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5b652-ff3d-4b9b-b2c9-c21f107a8227 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:38.440890 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5b652-ff3d-4b9b-b2c9-c21f107a8227 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:38.442784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5b652-ff3d-4b9b-b2c9-c21f107a8227 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:38.536465 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 563/1115] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:54:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:54:38.542445 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01469a1c-79a2-4fc2-b4b6-26bb3ba23dce None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:38.626964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:38.639463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01469a1c-79a2-4fc2-b4b6-26bb3ba23dce None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:38.673884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dba216-fda9-444f-acce-f6f340d24640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:38.674516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dba216-fda9-444f-acce-f6f340d24640 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:38.674795 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dba216-fda9-444f-acce-f6f340d24640 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:38.675067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dba216-fda9-444f-acce-f6f340d24640 None admin] PATH_INFO: `/v3/projects/3d132a17b1034d7f88bab67343c5a1e3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:38.676837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dba216-fda9-444f-acce-f6f340d24640 None admin] RBAC: Authorizing `identity:get_project(project_id=3d132a17b1034d7f88bab67343c5a1e3)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:38.678319 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dba216-fda9-444f-acce-f6f340d24640 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:38.679983 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 553/1116] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:54:38 2022] GET /identity/v3/projects/3d132a17b1034d7f88bab67343c5a1e3 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:38.686245 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5b652-ff3d-4b9b-b2c9-c21f107a8227 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:38.719748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:38.788555 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5b652-ff3d-4b9b-b2c9-c21f107a8227 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:38.821606 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa8a93-038a-483d-89d5-8cbbf3ec4fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:38.822233 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa8a93-038a-483d-89d5-8cbbf3ec4fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:38.822510 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa8a93-038a-483d-89d5-8cbbf3ec4fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:38.822780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa8a93-038a-483d-89d5-8cbbf3ec4fe6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:38.823488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa8a93-038a-483d-89d5-8cbbf3ec4fe6 None admin] RBAC: Adding query filter params (role.id=d43148a29472443ba7dfe2424ec60baf, scope.project.id=3d132a17b1034d7f88bab67343c5a1e3, user.id=7c2b6fe1904a4bdc93711b1a75a6602a) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:38.823790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa8a93-038a-483d-89d5-8cbbf3ec4fe6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:38.825195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa8a93-038a-483d-89d5-8cbbf3ec4fe6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:38.830403 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 564/1117] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:54:38 2022] GET /identity/v3/role_assignments?user.id=7c2b6fe1904a4bdc93711b1a75a6602a&scope.project.id=3d132a17b1034d7f88bab67343c5a1e3&role.id=d43148a29472443ba7dfe2424ec60baf => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:40.931159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fa448-197a-417c-a14d-6c28b89ee846 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:40.931700 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fa448-197a-417c-a14d-6c28b89ee846 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:40.931700 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fa448-197a-417c-a14d-6c28b89ee846 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:40.933004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 554/1118] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:40.938213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648e804-a30b-4619-8b44-95fd9019e8aa None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:40.938606 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648e804-a30b-4619-8b44-95fd9019e8aa None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:40.938760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648e804-a30b-4619-8b44-95fd9019e8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:40.957228 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a648e804-a30b-4619-8b44-95fd9019e8aa None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:40.958441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a648e804-a30b-4619-8b44-95fd9019e8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:41.024274 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 565/1119] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:41.291950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fa448-197a-417c-a14d-6c28b89ee846 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:41.390364 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fa448-197a-417c-a14d-6c28b89ee846 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:41.422947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdcca2-5567-4123-9954-4e247f754085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:41.423559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdcca2-5567-4123-9954-4e247f754085 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:41.423828 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdcca2-5567-4123-9954-4e247f754085 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:41.424091 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdcca2-5567-4123-9954-4e247f754085 None admin] PATH_INFO: `/v3/roles/d43148a29472443ba7dfe2424ec60baf` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:41.425485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdcca2-5567-4123-9954-4e247f754085 None admin] RBAC: Authorizing `identity:get_role(role_id=d43148a29472443ba7dfe2424ec60baf)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:41.426863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdcca2-5567-4123-9954-4e247f754085 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:41.428159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 555/1120] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:54:41 2022] GET /identity/v3/roles/d43148a29472443ba7dfe2424ec60baf => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:41.434043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648e804-a30b-4619-8b44-95fd9019e8aa None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:41.460944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:41.531552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648e804-a30b-4619-8b44-95fd9019e8aa None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:41.563518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc15f06-d7be-46c3-bfe1-c7b42865fa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:41.564147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc15f06-d7be-46c3-bfe1-c7b42865fa16 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:41.564425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc15f06-d7be-46c3-bfe1-c7b42865fa16 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:41.564697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc15f06-d7be-46c3-bfe1-c7b42865fa16 None admin] PATH_INFO: `/v3/users/7c2b6fe1904a4bdc93711b1a75a6602a` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:41.565813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc15f06-d7be-46c3-bfe1-c7b42865fa16 None admin] RBAC: Authorizing `identity:get_user(user_id=7c2b6fe1904a4bdc93711b1a75a6602a)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:41.567811 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc15f06-d7be-46c3-bfe1-c7b42865fa16 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:41.569393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 566/1121] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:54:41 2022] GET /identity/v3/users/7c2b6fe1904a4bdc93711b1a75a6602a => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:41.575233 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdcca2-5567-4123-9954-4e247f754085 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:41.600182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:41.671876 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdcca2-5567-4123-9954-4e247f754085 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:41.705338 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa11fa4-c0f8-4302-9ae5-b07f0d876e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:41.705947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa11fa4-c0f8-4302-9ae5-b07f0d876e01 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:41.706252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa11fa4-c0f8-4302-9ae5-b07f0d876e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:41.706523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa11fa4-c0f8-4302-9ae5-b07f0d876e01 None admin] PATH_INFO: `/v3/projects/3d132a17b1034d7f88bab67343c5a1e3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:41.707651 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa11fa4-c0f8-4302-9ae5-b07f0d876e01 None admin] RBAC: Authorizing `identity:get_project(project_id=3d132a17b1034d7f88bab67343c5a1e3)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:41.709071 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa11fa4-c0f8-4302-9ae5-b07f0d876e01 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:41.710714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 556/1122] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:54:41 2022] GET /identity/v3/projects/3d132a17b1034d7f88bab67343c5a1e3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:41.716712 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc15f06-d7be-46c3-bfe1-c7b42865fa16 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:41.808348 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:41.815295 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc15f06-d7be-46c3-bfe1-c7b42865fa16 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:41.848014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c563e-6afa-4462-ba53-b0c1cc135b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:41.848666 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c563e-6afa-4462-ba53-b0c1cc135b15 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:41.848946 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c563e-6afa-4462-ba53-b0c1cc135b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:41.849222 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c563e-6afa-4462-ba53-b0c1cc135b15 None admin] PATH_INFO: `/v3/projects/3d132a17b1034d7f88bab67343c5a1e3/users/7c2b6fe1904a4bdc93711b1a75a6602a/roles/d43148a29472443ba7dfe2424ec60baf` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:41.851388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c563e-6afa-4462-ba53-b0c1cc135b15 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d132a17b1034d7f88bab67343c5a1e3, user_id=7c2b6fe1904a4bdc93711b1a75a6602a, role_id=d43148a29472443ba7dfe2424ec60baf)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:41.852824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c563e-6afa-4462-ba53-b0c1cc135b15 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:41.866440 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 567/1123] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:54:41 2022] PUT /identity/v3/projects/3d132a17b1034d7f88bab67343c5a1e3/users/7c2b6fe1904a4bdc93711b1a75a6602a/roles/d43148a29472443ba7dfe2424ec60baf => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:44.005934 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35158a5-8b98-47f8-bc55-1669364f715b None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:44.006533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35158a5-8b98-47f8-bc55-1669364f715b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:44.006912 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35158a5-8b98-47f8-bc55-1669364f715b None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:44.009082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 557/1124] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:44.014499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fb1cb-b980-4151-8d95-e7a4592c916d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:44.014912 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fb1cb-b980-4151-8d95-e7a4592c916d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:44.015068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fb1cb-b980-4151-8d95-e7a4592c916d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:44.033523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-011fb1cb-b980-4151-8d95-e7a4592c916d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:44.034784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-011fb1cb-b980-4151-8d95-e7a4592c916d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:44.099335 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 568/1125] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:44.371393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35158a5-8b98-47f8-bc55-1669364f715b None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:44.463166 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:44.472890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35158a5-8b98-47f8-bc55-1669364f715b None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:44.506082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce725e5-fc96-498b-8e6b-f79bee866fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:44.506722 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce725e5-fc96-498b-8e6b-f79bee866fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:44.506995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce725e5-fc96-498b-8e6b-f79bee866fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:44.507261 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce725e5-fc96-498b-8e6b-f79bee866fd0 None admin] PATH_INFO: `/v3/roles/d43148a29472443ba7dfe2424ec60baf` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:44.508627 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce725e5-fc96-498b-8e6b-f79bee866fd0 None admin] RBAC: Authorizing `identity:get_role(role_id=d43148a29472443ba7dfe2424ec60baf)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:44.509980 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce725e5-fc96-498b-8e6b-f79bee866fd0 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:44.511316 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 558/1126] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:54:44 2022] GET /identity/v3/roles/d43148a29472443ba7dfe2424ec60baf => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:44.517279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fb1cb-b980-4151-8d95-e7a4592c916d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:44.596474 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:44.612934 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fb1cb-b980-4151-8d95-e7a4592c916d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:44.645892 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4df94-efed-410f-82b9-32c390e7dc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:44.646477 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4df94-efed-410f-82b9-32c390e7dc17 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:44.646754 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4df94-efed-410f-82b9-32c390e7dc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:44.647021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4df94-efed-410f-82b9-32c390e7dc17 None admin] PATH_INFO: `/v3/users/7c2b6fe1904a4bdc93711b1a75a6602a` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:44.648126 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a4df94-efed-410f-82b9-32c390e7dc17 None admin] RBAC: Authorizing `identity:get_user(user_id=7c2b6fe1904a4bdc93711b1a75a6602a)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:44.649549 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a4df94-efed-410f-82b9-32c390e7dc17 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:44.651140 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 569/1127] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:54:44 2022] GET /identity/v3/users/7c2b6fe1904a4bdc93711b1a75a6602a => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:44.658393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce725e5-fc96-498b-8e6b-f79bee866fd0 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:44.758847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce725e5-fc96-498b-8e6b-f79bee866fd0 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:44.791707 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2502cc-a97b-4c8f-a7da-adad16e2314f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:44.792305 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2502cc-a97b-4c8f-a7da-adad16e2314f None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:44.792583 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2502cc-a97b-4c8f-a7da-adad16e2314f None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:44.792853 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2502cc-a97b-4c8f-a7da-adad16e2314f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:44.820389 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:44.859570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2502cc-a97b-4c8f-a7da-adad16e2314f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:44.861006 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2502cc-a97b-4c8f-a7da-adad16e2314f None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:44.957881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 559/1128] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:54:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:54:44.963909 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4df94-efed-410f-82b9-32c390e7dc17 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:45.063722 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4df94-efed-410f-82b9-32c390e7dc17 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:45.095756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed25385-9625-41a3-a87c-40ad43cf6ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:45.096372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed25385-9625-41a3-a87c-40ad43cf6ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:45.096642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed25385-9625-41a3-a87c-40ad43cf6ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:45.096906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed25385-9625-41a3-a87c-40ad43cf6ec8 None admin] PATH_INFO: `/v3/projects/3d132a17b1034d7f88bab67343c5a1e3` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:45.098548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed25385-9625-41a3-a87c-40ad43cf6ec8 None admin] RBAC: Authorizing `identity:get_project(project_id=3d132a17b1034d7f88bab67343c5a1e3)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:45.099944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed25385-9625-41a3-a87c-40ad43cf6ec8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:45.101554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 570/1129] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:54:44 2022] GET /identity/v3/projects/3d132a17b1034d7f88bab67343c5a1e3 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:45.107881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2502cc-a97b-4c8f-a7da-adad16e2314f None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:45.165364 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:45.205757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2502cc-a97b-4c8f-a7da-adad16e2314f None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:45.239949 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68739286-26d0-4ae3-83b8-d3cf86127b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:45.240535 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739286-26d0-4ae3-83b8-d3cf86127b69 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:45.240807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739286-26d0-4ae3-83b8-d3cf86127b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:45.241072 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739286-26d0-4ae3-83b8-d3cf86127b69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:45.241785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68739286-26d0-4ae3-83b8-d3cf86127b69 None admin] RBAC: Adding query filter params (role.id=d43148a29472443ba7dfe2424ec60baf, scope.project.id=3d132a17b1034d7f88bab67343c5a1e3, user.id=7c2b6fe1904a4bdc93711b1a75a6602a) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:45.242089 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68739286-26d0-4ae3-83b8-d3cf86127b69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:45.243530 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68739286-26d0-4ae3-83b8-d3cf86127b69 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:45.249145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 560/1130] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:54:45 2022] GET /identity/v3/role_assignments?user.id=7c2b6fe1904a4bdc93711b1a75a6602a&scope.project.id=3d132a17b1034d7f88bab67343c5a1e3&role.id=d43148a29472443ba7dfe2424ec60baf => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:47.425273 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af99235-b9a1-47c7-a2b7-da53c6b98df0 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:47.426437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af99235-b9a1-47c7-a2b7-da53c6b98df0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:47.426534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af99235-b9a1-47c7-a2b7-da53c6b98df0 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:47.427876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 571/1131] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:47.433030 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b98a7-98d0-4d6a-a6be-d4e6c95529b4 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:47.433418 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b98a7-98d0-4d6a-a6be-d4e6c95529b4 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:47.433573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b98a7-98d0-4d6a-a6be-d4e6c95529b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:47.451447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-b83b98a7-98d0-4d6a-a6be-d4e6c95529b4 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:47.452630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b83b98a7-98d0-4d6a-a6be-d4e6c95529b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:47.517863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 561/1132] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:47.803422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af99235-b9a1-47c7-a2b7-da53c6b98df0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:47.825762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:47.905013 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af99235-b9a1-47c7-a2b7-da53c6b98df0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:47.937917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d10226-9270-4bc5-a710-646d508634c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:47.938565 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d10226-9270-4bc5-a710-646d508634c8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:47.938844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d10226-9270-4bc5-a710-646d508634c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:47.939113 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d10226-9270-4bc5-a710-646d508634c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:47.940219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d10226-9270-4bc5-a710-646d508634c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:47.942265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d10226-9270-4bc5-a710-646d508634c8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:47.943838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 572/1133] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:47.949837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b98a7-98d0-4d6a-a6be-d4e6c95529b4 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:48.003574 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:48.047459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b98a7-98d0-4d6a-a6be-d4e6c95529b4 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:48.080759 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5198c-2751-4206-b5af-a33c73f07a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:48.081359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5198c-2751-4206-b5af-a33c73f07a0c None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:48.082209 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5198c-2751-4206-b5af-a33c73f07a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:48.082480 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5198c-2751-4206-b5af-a33c73f07a0c None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:48.083115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5198c-2751-4206-b5af-a33c73f07a0c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:48.084530 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5198c-2751-4206-b5af-a33c73f07a0c None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:48.110411 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 562/1134] 10.222.0.62 () {60 vars in 1141 bytes} [Fri Sep 9 11:54:47 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:50.259060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742537b-f050-4565-86bb-23b9923d5d78 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:50.259576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742537b-f050-4565-86bb-23b9923d5d78 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:50.259576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742537b-f050-4565-86bb-23b9923d5d78 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:50.260954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 573/1135] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:50.266669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67b620-2c38-41b0-a353-0602ef29384e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:50.267078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67b620-2c38-41b0-a353-0602ef29384e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:50.267221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67b620-2c38-41b0-a353-0602ef29384e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:50.285644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-cf67b620-2c38-41b0-a353-0602ef29384e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:50.286871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-cf67b620-2c38-41b0-a353-0602ef29384e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:50.351764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 563/1136] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:50.619952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742537b-f050-4565-86bb-23b9923d5d78 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:50.706267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:50.719629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742537b-f050-4565-86bb-23b9923d5d78 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:50.753393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9429277-8174-4226-baa6-be031185ecef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:50.754019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9429277-8174-4226-baa6-be031185ecef None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:50.754326 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9429277-8174-4226-baa6-be031185ecef None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:50.754600 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9429277-8174-4226-baa6-be031185ecef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:50.755703 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9429277-8174-4226-baa6-be031185ecef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:50.757114 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9429277-8174-4226-baa6-be031185ecef None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:50.758704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 574/1137] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:54:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:50.764598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67b620-2c38-41b0-a353-0602ef29384e None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:50.798551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:50.862302 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67b620-2c38-41b0-a353-0602ef29384e None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:50.896179 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b7cb1-faf9-4c6c-b70d-6421a2462403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:50.896698 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b7cb1-faf9-4c6c-b70d-6421a2462403 None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:50.896985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b7cb1-faf9-4c6c-b70d-6421a2462403 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:50.897256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b7cb1-faf9-4c6c-b70d-6421a2462403 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:50.897909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b7cb1-faf9-4c6c-b70d-6421a2462403 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:50.899382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b7cb1-faf9-4c6c-b70d-6421a2462403 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:50.919728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 564/1138] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:54:50 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:54:53.077012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67276cd0-117a-4a82-a7b2-440c005cdf00 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:53.077554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67276cd0-117a-4a82-a7b2-440c005cdf00 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:53.077554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67276cd0-117a-4a82-a7b2-440c005cdf00 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:53.078913 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 575/1139] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:53.084137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983570ff-d2c4-4f07-b371-e94bc4f1ae7a None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:53.084518 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983570ff-d2c4-4f07-b371-e94bc4f1ae7a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:53.084687 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983570ff-d2c4-4f07-b371-e94bc4f1ae7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:53.102602 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-983570ff-d2c4-4f07-b371-e94bc4f1ae7a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:53.103790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-983570ff-d2c4-4f07-b371-e94bc4f1ae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:53.172520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 565/1140] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:53.464611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67276cd0-117a-4a82-a7b2-440c005cdf00 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:53.566997 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67276cd0-117a-4a82-a7b2-440c005cdf00 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:53.600803 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36d0cb-3d78-4f5e-87e1-537185597b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:53.601429 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36d0cb-3d78-4f5e-87e1-537185597b2e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:53.601704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36d0cb-3d78-4f5e-87e1-537185597b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:53.601984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36d0cb-3d78-4f5e-87e1-537185597b2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:53.612121 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36d0cb-3d78-4f5e-87e1-537185597b2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-ad36d0cb-3d78-4f5e-87e1-537185597b2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:54:53.617092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:53.618259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:53.619309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:53.619309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:53.619309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:53.619309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:53.619309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:54:53.619309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:54:53.619309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:54:53.619309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:54:53.619309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:54:53.619309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:54:53.619309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:54:53.619309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:53.619309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 576/1141] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:54:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:53.625196 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983570ff-d2c4-4f07-b371-e94bc4f1ae7a None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:53.648954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:53.725125 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983570ff-d2c4-4f07-b371-e94bc4f1ae7a None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:53.759940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa4417-bf5d-4281-a9ca-4e7be0f8f6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:53.760574 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa4417-bf5d-4281-a9ca-4e7be0f8f6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:53.761078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa4417-bf5d-4281-a9ca-4e7be0f8f6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:53.761573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa4417-bf5d-4281-a9ca-4e7be0f8f6a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:53.762365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa4417-bf5d-4281-a9ca-4e7be0f8f6a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:53.762673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa4417-bf5d-4281-a9ca-4e7be0f8f6a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:53.764035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa4417-bf5d-4281-a9ca-4e7be0f8f6a5 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:53.772634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 566/1142] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:54:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:53.778656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36d0cb-3d78-4f5e-87e1-537185597b2e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:53.797539 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:53.877259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36d0cb-3d78-4f5e-87e1-537185597b2e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:53.911508 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abcc995-80de-4c25-af34-96ff1dcd0b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:53.912131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcc995-80de-4c25-af34-96ff1dcd0b7c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:53.912405 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcc995-80de-4c25-af34-96ff1dcd0b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:53.912672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcc995-80de-4c25-af34-96ff1dcd0b7c None admin] PATH_INFO: `/v3/users/2859075a53774f7a99f0ea77f92cb963` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:53.926641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abcc995-80de-4c25-af34-96ff1dcd0b7c None admin] RBAC: Authorizing `identity:get_user(user_id=2859075a53774f7a99f0ea77f92cb963)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:53.928075 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abcc995-80de-4c25-af34-96ff1dcd0b7c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:53.929669 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 577/1143] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:54:53 2022] GET /identity/v3/users/2859075a53774f7a99f0ea77f92cb963 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:53.935666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa4417-bf5d-4281-a9ca-4e7be0f8f6a5 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:54.018313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:54.035011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa4417-bf5d-4281-a9ca-4e7be0f8f6a5 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:54.068474 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fddee-b8fa-435c-bfbb-ba6d09792a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:54.069093 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fddee-b8fa-435c-bfbb-ba6d09792a69 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:54.069369 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fddee-b8fa-435c-bfbb-ba6d09792a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:54.069642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fddee-b8fa-435c-bfbb-ba6d09792a69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:54.145092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3fddee-b8fa-435c-bfbb-ba6d09792a69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:54.146562 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3fddee-b8fa-435c-bfbb-ba6d09792a69 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:54.230193 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:54.248062 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 567/1144] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:54:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:54:54.254493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abcc995-80de-4c25-af34-96ff1dcd0b7c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:54.325118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:54.349355 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abcc995-80de-4c25-af34-96ff1dcd0b7c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:54.382664 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d935484-1228-414a-bb40-809965b08cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:54.383285 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d935484-1228-414a-bb40-809965b08cbb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:54.383556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d935484-1228-414a-bb40-809965b08cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:54.383819 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d935484-1228-414a-bb40-809965b08cbb None admin] PATH_INFO: `/v3/projects/9e681ad8bfe04d1bb94e39304bcd90f4` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:54.385047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d935484-1228-414a-bb40-809965b08cbb None admin] RBAC: Authorizing `identity:get_project(project_id=9e681ad8bfe04d1bb94e39304bcd90f4)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:54.386663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d935484-1228-414a-bb40-809965b08cbb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:54.388378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 578/1145] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:54:54 2022] GET /identity/v3/projects/9e681ad8bfe04d1bb94e39304bcd90f4 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:54.395444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fddee-b8fa-435c-bfbb-ba6d09792a69 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:54.495599 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fddee-b8fa-435c-bfbb-ba6d09792a69 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:54.529453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2e5cc-9e40-4a7f-9797-82f17a101e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:54.530040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2e5cc-9e40-4a7f-9797-82f17a101e9d None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:54.530339 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2e5cc-9e40-4a7f-9797-82f17a101e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:54.530603 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2e5cc-9e40-4a7f-9797-82f17a101e9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:54.531317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2e5cc-9e40-4a7f-9797-82f17a101e9d None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=9e681ad8bfe04d1bb94e39304bcd90f4, user.id=2859075a53774f7a99f0ea77f92cb963) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:54.531618 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2e5cc-9e40-4a7f-9797-82f17a101e9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:54.533033 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2e5cc-9e40-4a7f-9797-82f17a101e9d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:54.538986 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 568/1146] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:54:54 2022] GET /identity/v3/role_assignments?user.id=2859075a53774f7a99f0ea77f92cb963&scope.project.id=9e681ad8bfe04d1bb94e39304bcd90f4&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:54:56.638772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c294dd6-1093-407e-a7a2-695695a831eb None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:56.639361 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c294dd6-1093-407e-a7a2-695695a831eb None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:56.639361 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c294dd6-1093-407e-a7a2-695695a831eb None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:56.640645 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 579/1147] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:56.646217 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9737f-aebf-4e30-b812-72b75376d18b None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:56.646673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9737f-aebf-4e30-b812-72b75376d18b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:56.646786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9737f-aebf-4e30-b812-72b75376d18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:56.664596 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-4fd9737f-aebf-4e30-b812-72b75376d18b None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:56.665787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-4fd9737f-aebf-4e30-b812-72b75376d18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:56.708515 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:56.731587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 569/1148] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:54:57.000945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c294dd6-1093-407e-a7a2-695695a831eb None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:57.103811 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c294dd6-1093-407e-a7a2-695695a831eb None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:57.137291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2163a0-b844-4545-ae24-5dc37c92099e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:57.137919 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2163a0-b844-4545-ae24-5dc37c92099e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:57.138224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2163a0-b844-4545-ae24-5dc37c92099e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:57.138497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2163a0-b844-4545-ae24-5dc37c92099e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:57.149423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2163a0-b844-4545-ae24-5dc37c92099e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-eb2163a0-b844-4545-ae24-5dc37c92099e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:54:57.154268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:54:57.155433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:54:57.156363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:54:57.156363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:54:57.156363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:54:57.156363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:54:57.156363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:54:57.156363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:54:57.156363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:54:57.156363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:54:57.156363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:54:57.156363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:54:57.156363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:54:57.156363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:54:57.156363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 580/1149] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:54:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:54:57.162149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9737f-aebf-4e30-b812-72b75376d18b None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:57.263757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9737f-aebf-4e30-b812-72b75376d18b None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:57.296984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945aac77-f9a9-4fed-b8b2-d73b6fb91c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:57.297585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945aac77-f9a9-4fed-b8b2-d73b6fb91c02 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:57.297865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945aac77-f9a9-4fed-b8b2-d73b6fb91c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:57.298156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945aac77-f9a9-4fed-b8b2-d73b6fb91c02 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:57.298831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945aac77-f9a9-4fed-b8b2-d73b6fb91c02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:54:57.299133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945aac77-f9a9-4fed-b8b2-d73b6fb91c02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:57.301090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945aac77-f9a9-4fed-b8b2-d73b6fb91c02 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:57.309033 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 570/1150] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:54:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:57.314886 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2163a0-b844-4545-ae24-5dc37c92099e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:57.318330 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:57.414030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2163a0-b844-4545-ae24-5dc37c92099e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:57.447421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fa14a-626f-4ea2-a1f4-bb4041600a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:57.448038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fa14a-626f-4ea2-a1f4-bb4041600a4b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:57.448315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fa14a-626f-4ea2-a1f4-bb4041600a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:57.448588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fa14a-626f-4ea2-a1f4-bb4041600a4b None admin] PATH_INFO: `/v3/users/2859075a53774f7a99f0ea77f92cb963` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:57.449712 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fa14a-626f-4ea2-a1f4-bb4041600a4b None admin] RBAC: Authorizing `identity:get_user(user_id=2859075a53774f7a99f0ea77f92cb963)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:57.451163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fa14a-626f-4ea2-a1f4-bb4041600a4b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:57.452740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 581/1151] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:54:57 2022] GET /identity/v3/users/2859075a53774f7a99f0ea77f92cb963 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:57.460180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945aac77-f9a9-4fed-b8b2-d73b6fb91c02 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:57.501787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:57.560057 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945aac77-f9a9-4fed-b8b2-d73b6fb91c02 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:57.593154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198596c-db8e-4abe-b26e-b7d28fb44af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:57.593769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198596c-db8e-4abe-b26e-b7d28fb44af5 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:57.594040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198596c-db8e-4abe-b26e-b7d28fb44af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:57.594332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198596c-db8e-4abe-b26e-b7d28fb44af5 None admin] PATH_INFO: `/v3/projects/9e681ad8bfe04d1bb94e39304bcd90f4` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:57.595432 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198596c-db8e-4abe-b26e-b7d28fb44af5 None admin] RBAC: Authorizing `identity:get_project(project_id=9e681ad8bfe04d1bb94e39304bcd90f4)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:57.596884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198596c-db8e-4abe-b26e-b7d28fb44af5 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:57.598517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 571/1152] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:54:57 2022] GET /identity/v3/projects/9e681ad8bfe04d1bb94e39304bcd90f4 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:54:57.604449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fa14a-626f-4ea2-a1f4-bb4041600a4b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:54:57.671216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:54:57.702939 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fa14a-626f-4ea2-a1f4-bb4041600a4b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:54:57.736014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea82b2-4bf1-47ff-8b4b-458e04e091e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:54:57.736661 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea82b2-4bf1-47ff-8b4b-458e04e091e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:57.736935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea82b2-4bf1-47ff-8b4b-458e04e091e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:57.737203 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea82b2-4bf1-47ff-8b4b-458e04e091e8 None admin] PATH_INFO: `/v3/projects/9e681ad8bfe04d1bb94e39304bcd90f4/users/2859075a53774f7a99f0ea77f92cb963/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:57.739606 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea82b2-4bf1-47ff-8b4b-458e04e091e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e681ad8bfe04d1bb94e39304bcd90f4, user_id=2859075a53774f7a99f0ea77f92cb963, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:54:57.741146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea82b2-4bf1-47ff-8b4b-458e04e091e8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:54:57.755259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 582/1153] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:54:57 2022] PUT /identity/v3/projects/9e681ad8bfe04d1bb94e39304bcd90f4/users/2859075a53774f7a99f0ea77f92cb963/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:54:59.905871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbeaf1-c3e0-4d11-ab3b-671c847e74d5 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:59.906462 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbeaf1-c3e0-4d11-ab3b-671c847e74d5 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:59.906462 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbeaf1-c3e0-4d11-ab3b-671c847e74d5 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:59.907784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 572/1154] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:54:59.913546 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b57483-8d62-4e7a-884e-0b29740397c6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:54:59.913968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b57483-8d62-4e7a-884e-0b29740397c6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:54:59.914091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b57483-8d62-4e7a-884e-0b29740397c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:54:59.932159 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-44b57483-8d62-4e7a-884e-0b29740397c6 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:54:59.933350 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-44b57483-8d62-4e7a-884e-0b29740397c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:54:59.999584 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 583/1155] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:54:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:55:00.296056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbeaf1-c3e0-4d11-ab3b-671c847e74d5 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:00.398568 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbeaf1-c3e0-4d11-ab3b-671c847e74d5 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:00.407141 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:00.431997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d75fd-29ce-428e-b101-6735aabbbf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:00.432614 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d75fd-29ce-428e-b101-6735aabbbf90 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:00.432890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d75fd-29ce-428e-b101-6735aabbbf90 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:00.433156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d75fd-29ce-428e-b101-6735aabbbf90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:00.444236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d75fd-29ce-428e-b101-6735aabbbf90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-8a1d75fd-29ce-428e-b101-6735aabbbf90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:55:00.449144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:55:00.450296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:55:00.451270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:55:00.451270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:55:00.451270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:55:00.451270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:55:00.451270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:55:00.451270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:55:00.451270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:55:00.451270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:55:00.451270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:55:00.451270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:55:00.451270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:55:00.451270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:55:00.451270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 573/1156] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:55:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:55:00.456471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b57483-8d62-4e7a-884e-0b29740397c6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:00.513627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:00.554251 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b57483-8d62-4e7a-884e-0b29740397c6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:00.604597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7afa58-d34a-4c8a-9b08-529882b7e7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:00.605220 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7afa58-d34a-4c8a-9b08-529882b7e7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:00.605495 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7afa58-d34a-4c8a-9b08-529882b7e7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:00.605766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7afa58-d34a-4c8a-9b08-529882b7e7d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:00.606469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7afa58-d34a-4c8a-9b08-529882b7e7d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:55:00.606770 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7afa58-d34a-4c8a-9b08-529882b7e7d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:00.608449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7afa58-d34a-4c8a-9b08-529882b7e7d5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:00.619822 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 584/1157] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:55:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:00.626047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d75fd-29ce-428e-b101-6735aabbbf90 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:00.739111 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d75fd-29ce-428e-b101-6735aabbbf90 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:00.772785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be2b1c-6683-4cd8-9d98-32c42d11d715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:00.773356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be2b1c-6683-4cd8-9d98-32c42d11d715 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:00.773629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be2b1c-6683-4cd8-9d98-32c42d11d715 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:00.773896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be2b1c-6683-4cd8-9d98-32c42d11d715 None admin] PATH_INFO: `/v3/users/2859075a53774f7a99f0ea77f92cb963` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:00.775047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be2b1c-6683-4cd8-9d98-32c42d11d715 None admin] RBAC: Authorizing `identity:get_user(user_id=2859075a53774f7a99f0ea77f92cb963)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:00.776466 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be2b1c-6683-4cd8-9d98-32c42d11d715 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:00.778042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 574/1158] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:55:00 2022] GET /identity/v3/users/2859075a53774f7a99f0ea77f92cb963 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:00.784000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7afa58-d34a-4c8a-9b08-529882b7e7d5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:00.888023 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7afa58-d34a-4c8a-9b08-529882b7e7d5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:00.910207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:00.922052 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c54d11-3019-4be3-af04-2ecbca725655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:00.922693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c54d11-3019-4be3-af04-2ecbca725655 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:00.922969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c54d11-3019-4be3-af04-2ecbca725655 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:00.923236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c54d11-3019-4be3-af04-2ecbca725655 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:00.989121 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c54d11-3019-4be3-af04-2ecbca725655 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:00.990581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c54d11-3019-4be3-af04-2ecbca725655 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:01.083966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 585/1159] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:55:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:55:01.089932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be2b1c-6683-4cd8-9d98-32c42d11d715 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:01.127384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:01.190823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be2b1c-6683-4cd8-9d98-32c42d11d715 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:01.225350 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418719a9-ac39-4793-924b-bf31bf7cdca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:01.225485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418719a9-ac39-4793-924b-bf31bf7cdca9 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:01.225706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418719a9-ac39-4793-924b-bf31bf7cdca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:01.226005 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418719a9-ac39-4793-924b-bf31bf7cdca9 None admin] PATH_INFO: `/v3/projects/9e681ad8bfe04d1bb94e39304bcd90f4` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:01.227168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418719a9-ac39-4793-924b-bf31bf7cdca9 None admin] RBAC: Authorizing `identity:get_project(project_id=9e681ad8bfe04d1bb94e39304bcd90f4)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:01.228605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418719a9-ac39-4793-924b-bf31bf7cdca9 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:01.230252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 575/1160] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:55:01 2022] GET /identity/v3/projects/9e681ad8bfe04d1bb94e39304bcd90f4 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:01.236532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c54d11-3019-4be3-af04-2ecbca725655 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:01.259563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:01.333038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c54d11-3019-4be3-af04-2ecbca725655 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:01.366626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b32de8-35dd-4e4c-b398-422f6e2c7113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:01.367158 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b32de8-35dd-4e4c-b398-422f6e2c7113 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:01.367435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b32de8-35dd-4e4c-b398-422f6e2c7113 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:01.367708 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b32de8-35dd-4e4c-b398-422f6e2c7113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:01.368419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b32de8-35dd-4e4c-b398-422f6e2c7113 None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=9e681ad8bfe04d1bb94e39304bcd90f4, user.id=2859075a53774f7a99f0ea77f92cb963) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:55:01.368723 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b32de8-35dd-4e4c-b398-422f6e2c7113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:01.370155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b32de8-35dd-4e4c-b398-422f6e2c7113 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:01.375808 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 586/1161] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:55:01 2022] GET /identity/v3/role_assignments?user.id=2859075a53774f7a99f0ea77f92cb963&scope.project.id=9e681ad8bfe04d1bb94e39304bcd90f4&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:03.553353 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb83a7-8162-4c37-9f05-96e88382f38f None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:03.553880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb83a7-8162-4c37-9f05-96e88382f38f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:03.553880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb83a7-8162-4c37-9f05-96e88382f38f None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:03.555290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 576/1162] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:55:03.560395 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4fcb9-adc8-43c0-8811-a9d9ccd40605 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:03.560834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4fcb9-adc8-43c0-8811-a9d9ccd40605 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:03.560937 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4fcb9-adc8-43c0-8811-a9d9ccd40605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:03.579481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-90e4fcb9-adc8-43c0-8811-a9d9ccd40605 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:03.580662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-90e4fcb9-adc8-43c0-8811-a9d9ccd40605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:03.645041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 587/1163] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:55:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:55:03.917410 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb83a7-8162-4c37-9f05-96e88382f38f None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:04.018185 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb83a7-8162-4c37-9f05-96e88382f38f None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:04.021004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:04.051163 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849e7fa-4eda-42d8-ba09-9e28d5f3f351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:04.051750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849e7fa-4eda-42d8-ba09-9e28d5f3f351 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:04.052025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849e7fa-4eda-42d8-ba09-9e28d5f3f351 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:04.052293 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849e7fa-4eda-42d8-ba09-9e28d5f3f351 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:04.116761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849e7fa-4eda-42d8-ba09-9e28d5f3f351 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:04.118221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849e7fa-4eda-42d8-ba09-9e28d5f3f351 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:04.220948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 577/1164] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:55:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:55:04.227102 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4fcb9-adc8-43c0-8811-a9d9ccd40605 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:04.245114 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:04.326848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4fcb9-adc8-43c0-8811-a9d9ccd40605 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:04.363171 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef15b5-ae42-46ea-bb36-252de5918bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:04.363794 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef15b5-ae42-46ea-bb36-252de5918bbd None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:04.364065 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef15b5-ae42-46ea-bb36-252de5918bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:04.364326 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef15b5-ae42-46ea-bb36-252de5918bbd None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:04.370815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef15b5-ae42-46ea-bb36-252de5918bbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:04.372224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef15b5-ae42-46ea-bb36-252de5918bbd None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-03ef15b5-ae42-46ea-bb36-252de5918bbd None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:55:04.383875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:55:04.385033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:55:04.385922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:55:04.385922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:55:04.385922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:55:04.385922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:55:04.385922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:55:04.385922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:55:04.385922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:55:04.385922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:55:04.385922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:55:04.385922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:55:04.385922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 09 11:55:04.385922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:55:04.385922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 11:55:04.385922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:55:04.385922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 588/1165] 10.222.0.62 () {56 vars in 1117 bytes} [Fri Sep 9 11:55:04 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:55:04.390980 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849e7fa-4eda-42d8-ba09-9e28d5f3f351 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:04.394215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:04.487529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849e7fa-4eda-42d8-ba09-9e28d5f3f351 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:04.520448 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f19a0-c9c2-454e-8a00-e3629fe287f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:04.521033 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f19a0-c9c2-454e-8a00-e3629fe287f0 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:04.521306 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f19a0-c9c2-454e-8a00-e3629fe287f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:04.521571 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f19a0-c9c2-454e-8a00-e3629fe287f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:04.522225 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f19a0-c9c2-454e-8a00-e3629fe287f0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:55:04.522523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f19a0-c9c2-454e-8a00-e3629fe287f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:04.523865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f19a0-c9c2-454e-8a00-e3629fe287f0 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:04.530756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 578/1166] 10.222.0.62 () {56 vars in 1115 bytes} [Fri Sep 9 11:55:04 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:06.671291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9e61c-69b5-4964-8cd1-554a2cced384 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:06.671870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9e61c-69b5-4964-8cd1-554a2cced384 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:06.672216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9e61c-69b5-4964-8cd1-554a2cced384 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:06.675222 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 589/1167] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:55:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:55:06.681062 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cb959-48b7-4753-acd3-80e31d6f0dcf None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:06.681607 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cb959-48b7-4753-acd3-80e31d6f0dcf None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:06.681607 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cb959-48b7-4753-acd3-80e31d6f0dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:06.701286 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-872cb959-48b7-4753-acd3-80e31d6f0dcf None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:06.702547 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-872cb959-48b7-4753-acd3-80e31d6f0dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:06.770438 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:06.776731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 579/1168] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:55:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:55:07.045526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9e61c-69b5-4964-8cd1-554a2cced384 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:07.125476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:07.150331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9e61c-69b5-4964-8cd1-554a2cced384 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:07.184079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713384e1-0792-4955-81a7-99b5485e49ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:07.184661 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713384e1-0792-4955-81a7-99b5485e49ea None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:07.184930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713384e1-0792-4955-81a7-99b5485e49ea None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:07.185204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713384e1-0792-4955-81a7-99b5485e49ea None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:07.185871 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713384e1-0792-4955-81a7-99b5485e49ea None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:07.187264 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713384e1-0792-4955-81a7-99b5485e49ea None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:07.203852 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 590/1169] 10.222.0.62 () {60 vars in 1138 bytes} [Fri Sep 9 11:55:07 2022] POST /identity/v3/domains => generated 267 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:55:09.370616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17380262-c857-47e7-b403-cb020909abba None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:09.371149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17380262-c857-47e7-b403-cb020909abba None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:09.371149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17380262-c857-47e7-b403-cb020909abba None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:09.372498 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 580/1170] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:55:09.378003 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42420fed-aa95-4398-9bcc-4acefd270f6d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:09.378490 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42420fed-aa95-4398-9bcc-4acefd270f6d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:09.378593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42420fed-aa95-4398-9bcc-4acefd270f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:09.396849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-42420fed-aa95-4398-9bcc-4acefd270f6d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:09.398003 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-42420fed-aa95-4398-9bcc-4acefd270f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:09.468550 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 591/1171] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:55:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:55:09.747501 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17380262-c857-47e7-b403-cb020909abba None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:09.857363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17380262-c857-47e7-b403-cb020909abba None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:09.894964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6523eb2-ee86-43b5-9245-d1066fc8dbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:09.895591 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6523eb2-ee86-43b5-9245-d1066fc8dbce None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:09.895876 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6523eb2-ee86-43b5-9245-d1066fc8dbce None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:09.896141 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6523eb2-ee86-43b5-9245-d1066fc8dbce None admin] PATH_INFO: `/v3/domains/8416fde44c6240728ce6a7bab6db2e25` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:09.906532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6523eb2-ee86-43b5-9245-d1066fc8dbce None admin] RBAC: Authorizing `identity:get_domain(domain_id=8416fde44c6240728ce6a7bab6db2e25)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:09.907974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6523eb2-ee86-43b5-9245-d1066fc8dbce None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:09.909587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 581/1172] 10.222.0.62 () {56 vars in 1183 bytes} [Fri Sep 9 11:55:09 2022] GET /identity/v3/domains/8416fde44c6240728ce6a7bab6db2e25 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:09.915441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42420fed-aa95-4398-9bcc-4acefd270f6d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:09.979820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:10.014180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42420fed-aa95-4398-9bcc-4acefd270f6d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:10.046359 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e5e04-fad9-4997-8b25-4ebba2988e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:10.046959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e5e04-fad9-4997-8b25-4ebba2988e4a None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:10.047233 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e5e04-fad9-4997-8b25-4ebba2988e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:10.047501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e5e04-fad9-4997-8b25-4ebba2988e4a None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:10.048142 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e5e04-fad9-4997-8b25-4ebba2988e4a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:10.049583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e5e04-fad9-4997-8b25-4ebba2988e4a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:10.075485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 592/1173] 10.222.0.62 () {60 vars in 1141 bytes} [Fri Sep 9 11:55:09 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:55:12.245168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec89f2-748b-49e0-b9bc-da537539cb77 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:12.245619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec89f2-748b-49e0-b9bc-da537539cb77 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:12.245721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec89f2-748b-49e0-b9bc-da537539cb77 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:12.247066 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 582/1174] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:55:12.252410 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a944e1b-db3d-44f5-adae-5ef913532531 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:12.252822 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a944e1b-db3d-44f5-adae-5ef913532531 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:12.252962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a944e1b-db3d-44f5-adae-5ef913532531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:12.270797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7a944e1b-db3d-44f5-adae-5ef913532531 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:12.271979 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7a944e1b-db3d-44f5-adae-5ef913532531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:12.337131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 593/1175] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:55:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:55:12.604870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec89f2-748b-49e0-b9bc-da537539cb77 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:12.664360 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:12.707774 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec89f2-748b-49e0-b9bc-da537539cb77 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:12.740379 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf4fec-84d8-450f-b548-5bc97b9db399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:12.740993 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf4fec-84d8-450f-b548-5bc97b9db399 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:12.741267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf4fec-84d8-450f-b548-5bc97b9db399 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:12.741533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf4fec-84d8-450f-b548-5bc97b9db399 None admin] PATH_INFO: `/v3/domains/8416fde44c6240728ce6a7bab6db2e25` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:12.742652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf4fec-84d8-450f-b548-5bc97b9db399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8416fde44c6240728ce6a7bab6db2e25)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:12.744077 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf4fec-84d8-450f-b548-5bc97b9db399 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:12.745619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 583/1176] 10.222.0.62 () {56 vars in 1183 bytes} [Fri Sep 9 11:55:12 2022] GET /identity/v3/domains/8416fde44c6240728ce6a7bab6db2e25 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:12.751488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a944e1b-db3d-44f5-adae-5ef913532531 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:12.801558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:12.850666 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a944e1b-db3d-44f5-adae-5ef913532531 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:12.884472 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e514bc-53c1-40de-9483-41db820f1eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:12.884997 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e514bc-53c1-40de-9483-41db820f1eb3 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:12.885267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e514bc-53c1-40de-9483-41db820f1eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:12.885528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e514bc-53c1-40de-9483-41db820f1eb3 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:12.886191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e514bc-53c1-40de-9483-41db820f1eb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:12.887604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e514bc-53c1-40de-9483-41db820f1eb3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:12.909062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 594/1177] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:55:12 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:55:15.100510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b576113-5027-4ad6-8596-b5e9eaf0a526 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:15.101121 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b576113-5027-4ad6-8596-b5e9eaf0a526 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:15.101121 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b576113-5027-4ad6-8596-b5e9eaf0a526 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:15.102438 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 584/1178] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:55:15.108008 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d24d59-5810-448e-b37b-f1d31613d816 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:15.108412 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d24d59-5810-448e-b37b-f1d31613d816 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:15.108560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d24d59-5810-448e-b37b-f1d31613d816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:15.127012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-42d24d59-5810-448e-b37b-f1d31613d816 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:15.128139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-42d24d59-5810-448e-b37b-f1d31613d816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:15.192655 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 595/1179] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:55:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:55:15.467905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b576113-5027-4ad6-8596-b5e9eaf0a526 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:15.569633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b576113-5027-4ad6-8596-b5e9eaf0a526 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:15.598463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:15.607307 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b33c5-609e-4bae-b39e-a667fcd4aa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:15.608191 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b33c5-609e-4bae-b39e-a667fcd4aa64 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:15.608469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b33c5-609e-4bae-b39e-a667fcd4aa64 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:15.608738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b33c5-609e-4bae-b39e-a667fcd4aa64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:15.619367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b33c5-609e-4bae-b39e-a667fcd4aa64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-934b33c5-609e-4bae-b39e-a667fcd4aa64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:55:15.624180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:55:15.625291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:55:15.626345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:55:15.626345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:55:15.626345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:55:15.626345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:55:15.626345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:55:15.626345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:55:15.626345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:55:15.626345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:55:15.626345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:55:15.626345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:55:15.626345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:55:15.626345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:55:15.626345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 585/1180] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:55:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:55:15.631576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d24d59-5810-448e-b37b-f1d31613d816 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:15.664738 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:15.728680 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d24d59-5810-448e-b37b-f1d31613d816 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:15.760877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562c521-c522-4d44-9972-c875a55deb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:15.761475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562c521-c522-4d44-9972-c875a55deb83 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:15.761751 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562c521-c522-4d44-9972-c875a55deb83 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:15.762022 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562c521-c522-4d44-9972-c875a55deb83 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:15.762711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562c521-c522-4d44-9972-c875a55deb83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:55:15.763015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562c521-c522-4d44-9972-c875a55deb83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:15.764376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562c521-c522-4d44-9972-c875a55deb83 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:15.772748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 596/1181] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:55:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:15.778820 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b33c5-609e-4bae-b39e-a667fcd4aa64 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:15.876720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b33c5-609e-4bae-b39e-a667fcd4aa64 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:15.911823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75682f5e-7d39-43d5-9159-f3c798117e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:15.912444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75682f5e-7d39-43d5-9159-f3c798117e5c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:15.912724 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75682f5e-7d39-43d5-9159-f3c798117e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:15.912998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75682f5e-7d39-43d5-9159-f3c798117e5c None admin] PATH_INFO: `/v3/users/5221d5c2baa3491e969a208f18d93438` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:15.926071 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75682f5e-7d39-43d5-9159-f3c798117e5c None admin] RBAC: Authorizing `identity:get_user(user_id=5221d5c2baa3491e969a208f18d93438)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:15.927539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75682f5e-7d39-43d5-9159-f3c798117e5c None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:15.929130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 586/1182] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:55:15 2022] GET /identity/v3/users/5221d5c2baa3491e969a208f18d93438 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:15.935062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562c521-c522-4d44-9972-c875a55deb83 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:16.027448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:16.033885 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562c521-c522-4d44-9972-c875a55deb83 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:16.066427 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa498a2d-3a21-4df0-ae72-184bc07778e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:16.067028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa498a2d-3a21-4df0-ae72-184bc07778e0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:16.067307 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa498a2d-3a21-4df0-ae72-184bc07778e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:16.067578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa498a2d-3a21-4df0-ae72-184bc07778e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:16.132365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa498a2d-3a21-4df0-ae72-184bc07778e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:16.133799 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa498a2d-3a21-4df0-ae72-184bc07778e0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:16.225360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:16.231463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 597/1183] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:55:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:55:16.240642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75682f5e-7d39-43d5-9159-f3c798117e5c None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:16.286814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:16.338817 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75682f5e-7d39-43d5-9159-f3c798117e5c None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:16.372019 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473fc73-b340-4f11-951f-1709aa4dc6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:16.372633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473fc73-b340-4f11-951f-1709aa4dc6ea None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:16.372917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473fc73-b340-4f11-951f-1709aa4dc6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:16.373186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473fc73-b340-4f11-951f-1709aa4dc6ea None admin] PATH_INFO: `/v3/projects/b53a74f4aa094366a6811ccfa60d30f1` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:16.374321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473fc73-b340-4f11-951f-1709aa4dc6ea None admin] RBAC: Authorizing `identity:get_project(project_id=b53a74f4aa094366a6811ccfa60d30f1)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:16.375746 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473fc73-b340-4f11-951f-1709aa4dc6ea None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:16.377398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 587/1184] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:55:16 2022] GET /identity/v3/projects/b53a74f4aa094366a6811ccfa60d30f1 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:16.383541 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa498a2d-3a21-4df0-ae72-184bc07778e0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:16.478908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa498a2d-3a21-4df0-ae72-184bc07778e0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:16.511848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45512595-7949-4ad6-b377-b3b75edad4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:16.512434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45512595-7949-4ad6-b377-b3b75edad4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:16.512705 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45512595-7949-4ad6-b377-b3b75edad4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:16.512971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45512595-7949-4ad6-b377-b3b75edad4b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:16.513678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45512595-7949-4ad6-b377-b3b75edad4b1 None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=b53a74f4aa094366a6811ccfa60d30f1, user.id=5221d5c2baa3491e969a208f18d93438) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:55:16.513977 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45512595-7949-4ad6-b377-b3b75edad4b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:16.515404 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45512595-7949-4ad6-b377-b3b75edad4b1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:16.521080 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 598/1185] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:55:16 2022] GET /identity/v3/role_assignments?user.id=5221d5c2baa3491e969a208f18d93438&scope.project.id=b53a74f4aa094366a6811ccfa60d30f1&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:18.655688 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29cfc7-35bc-46a2-bf52-6eda1dde3560 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:18.656243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29cfc7-35bc-46a2-bf52-6eda1dde3560 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:18.656243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29cfc7-35bc-46a2-bf52-6eda1dde3560 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:18.657572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 588/1186] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:55:18.663136 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4cfd1-34e2-484b-ac30-09af578d0d70 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:18.663531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4cfd1-34e2-484b-ac30-09af578d0d70 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:18.663671 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4cfd1-34e2-484b-ac30-09af578d0d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:18.681895 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-36f4cfd1-34e2-484b-ac30-09af578d0d70 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:18.683106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-36f4cfd1-34e2-484b-ac30-09af578d0d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:18.736866 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:18.748316 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 599/1187] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:55:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:55:19.011991 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29cfc7-35bc-46a2-bf52-6eda1dde3560 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:19.112256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29cfc7-35bc-46a2-bf52-6eda1dde3560 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:19.125261 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:19.146336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c505d-7379-41c1-823e-76120ad52811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:19.146954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c505d-7379-41c1-823e-76120ad52811 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:19.147229 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c505d-7379-41c1-823e-76120ad52811 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:19.147497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c505d-7379-41c1-823e-76120ad52811 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:19.157378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c505d-7379-41c1-823e-76120ad52811 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-9f1c505d-7379-41c1-823e-76120ad52811 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:55:19.162811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:55:19.163917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:55:19.165142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:55:19.165142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:55:19.165142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:55:19.165142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:55:19.165142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:55:19.165142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:55:19.165142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:55:19.165142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:55:19.165142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:55:19.165142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:55:19.165142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:55:19.165142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:55:19.165142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 589/1188] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:55:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:55:19.169974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4cfd1-34e2-484b-ac30-09af578d0d70 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:19.264559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4cfd1-34e2-484b-ac30-09af578d0d70 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:19.297168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c020a67-9088-44a1-8680-ce43c74931ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:19.297766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c020a67-9088-44a1-8680-ce43c74931ed None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:19.298037 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c020a67-9088-44a1-8680-ce43c74931ed None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:19.298332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c020a67-9088-44a1-8680-ce43c74931ed None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:19.299009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c020a67-9088-44a1-8680-ce43c74931ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:55:19.299308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c020a67-9088-44a1-8680-ce43c74931ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:19.300641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c020a67-9088-44a1-8680-ce43c74931ed None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:19.308374 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 600/1189] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:55:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:19.314808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c505d-7379-41c1-823e-76120ad52811 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:19.413803 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c505d-7379-41c1-823e-76120ad52811 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:19.447225 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397b0b5-bcd4-4e92-ae06-8a121790d85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:19.447841 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397b0b5-bcd4-4e92-ae06-8a121790d85f None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:19.448115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397b0b5-bcd4-4e92-ae06-8a121790d85f None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:19.448387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397b0b5-bcd4-4e92-ae06-8a121790d85f None admin] PATH_INFO: `/v3/users/5221d5c2baa3491e969a208f18d93438` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:19.449502 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397b0b5-bcd4-4e92-ae06-8a121790d85f None admin] RBAC: Authorizing `identity:get_user(user_id=5221d5c2baa3491e969a208f18d93438)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:19.450945 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397b0b5-bcd4-4e92-ae06-8a121790d85f None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:19.452505 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 590/1190] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:55:19 2022] GET /identity/v3/users/5221d5c2baa3491e969a208f18d93438 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:19.458505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c020a67-9088-44a1-8680-ce43c74931ed None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:19.511165 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:19.557513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c020a67-9088-44a1-8680-ce43c74931ed None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:19.590077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ee561-78a2-4799-ac09-67a996790984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:19.590729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ee561-78a2-4799-ac09-67a996790984 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:19.591009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ee561-78a2-4799-ac09-67a996790984 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:19.591281 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ee561-78a2-4799-ac09-67a996790984 None admin] PATH_INFO: `/v3/projects/b53a74f4aa094366a6811ccfa60d30f1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:19.592394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ee561-78a2-4799-ac09-67a996790984 None admin] RBAC: Authorizing `identity:get_project(project_id=b53a74f4aa094366a6811ccfa60d30f1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:19.593824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ee561-78a2-4799-ac09-67a996790984 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:19.595504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 601/1191] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:55:19 2022] GET /identity/v3/projects/b53a74f4aa094366a6811ccfa60d30f1 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:19.602011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397b0b5-bcd4-4e92-ae06-8a121790d85f None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:19.644475 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:19.701395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397b0b5-bcd4-4e92-ae06-8a121790d85f None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:19.735582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f44789-f62b-4084-8bba-86992b584f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:19.736224 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f44789-f62b-4084-8bba-86992b584f14 None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:19.736501 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f44789-f62b-4084-8bba-86992b584f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:19.736773 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f44789-f62b-4084-8bba-86992b584f14 None admin] PATH_INFO: `/v3/projects/b53a74f4aa094366a6811ccfa60d30f1/users/5221d5c2baa3491e969a208f18d93438/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:19.738691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f44789-f62b-4084-8bba-86992b584f14 None admin] RBAC: Authorizing `identity:create_grant(project_id=b53a74f4aa094366a6811ccfa60d30f1, user_id=5221d5c2baa3491e969a208f18d93438, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:19.740132 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f44789-f62b-4084-8bba-86992b584f14 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:19.753457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 591/1192] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:55:19 2022] PUT /identity/v3/projects/b53a74f4aa094366a6811ccfa60d30f1/users/5221d5c2baa3491e969a208f18d93438/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:55:21.861905 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e06546-5bdf-4700-bede-78ac0aeacea9 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:21.862500 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e06546-5bdf-4700-bede-78ac0aeacea9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:21.862500 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e06546-5bdf-4700-bede-78ac0aeacea9 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:21.863823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 602/1193] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:55:21.869253 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebad8df-2501-4cdf-8535-da96e4c21d56 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:21.869723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebad8df-2501-4cdf-8535-da96e4c21d56 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:21.869819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebad8df-2501-4cdf-8535-da96e4c21d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:21.889919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-4ebad8df-2501-4cdf-8535-da96e4c21d56 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:55:21.891166 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-4ebad8df-2501-4cdf-8535-da96e4c21d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:55:21.958164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 592/1194] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:55:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:55:22.227628 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e06546-5bdf-4700-bede-78ac0aeacea9 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:22.326678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e06546-5bdf-4700-bede-78ac0aeacea9 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:22.345632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:22.359702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358455cf-a340-4a74-837d-8e318b011d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:22.360335 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358455cf-a340-4a74-837d-8e318b011d48 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:22.360615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358455cf-a340-4a74-837d-8e318b011d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:22.360884 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358455cf-a340-4a74-837d-8e318b011d48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:22.371652 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358455cf-a340-4a74-837d-8e318b011d48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-358455cf-a340-4a74-837d-8e318b011d48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:55:22.376542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:55:22.379259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:55:22.380824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:55:22.380824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:55:22.380824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:55:22.380824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:55:22.380824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:55:22.380824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:55:22.380824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:55:22.380824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:55:22.380824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:55:22.380824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:55:22.380824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:55:22.380824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:55:22.380824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 603/1195] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:55:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:55:22.384908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebad8df-2501-4cdf-8535-da96e4c21d56 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:22.413842 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:22.483420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebad8df-2501-4cdf-8535-da96e4c21d56 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:22.518464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5718be9-0893-4cc3-9fa0-576565ab7a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:22.519058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5718be9-0893-4cc3-9fa0-576565ab7a94 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:22.519333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5718be9-0893-4cc3-9fa0-576565ab7a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:22.519596 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5718be9-0893-4cc3-9fa0-576565ab7a94 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:22.520247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5718be9-0893-4cc3-9fa0-576565ab7a94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:55:22.520542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5718be9-0893-4cc3-9fa0-576565ab7a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:22.521910 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5718be9-0893-4cc3-9fa0-576565ab7a94 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:22.529796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 593/1196] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:55:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:22.536395 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358455cf-a340-4a74-837d-8e318b011d48 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:22.634434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358455cf-a340-4a74-837d-8e318b011d48 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:22.667839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a973d1-79e6-4a03-bc35-8933b5398f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:22.668461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a973d1-79e6-4a03-bc35-8933b5398f2b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:22.668736 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a973d1-79e6-4a03-bc35-8933b5398f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:22.669008 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a973d1-79e6-4a03-bc35-8933b5398f2b None admin] PATH_INFO: `/v3/users/5221d5c2baa3491e969a208f18d93438` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:22.670152 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a973d1-79e6-4a03-bc35-8933b5398f2b None admin] RBAC: Authorizing `identity:get_user(user_id=5221d5c2baa3491e969a208f18d93438)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:22.671586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a973d1-79e6-4a03-bc35-8933b5398f2b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:22.673150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 604/1197] 10.222.0.62 () {56 vars in 1177 bytes} [Fri Sep 9 11:55:22 2022] GET /identity/v3/users/5221d5c2baa3491e969a208f18d93438 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:22.679735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5718be9-0893-4cc3-9fa0-576565ab7a94 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:22.769763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:22.778647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5718be9-0893-4cc3-9fa0-576565ab7a94 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:22.811726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91be42d-d567-4dbe-a0c5-a858c99e9047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:22.812322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91be42d-d567-4dbe-a0c5-a858c99e9047 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:22.812594 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91be42d-d567-4dbe-a0c5-a858c99e9047 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:22.812861 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91be42d-d567-4dbe-a0c5-a858c99e9047 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:22.877728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91be42d-d567-4dbe-a0c5-a858c99e9047 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:22.879206 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91be42d-d567-4dbe-a0c5-a858c99e9047 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:22.968274 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:22.978216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 594/1198] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:55:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:55:22.984505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a973d1-79e6-4a03-bc35-8933b5398f2b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:23.031642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:23.080824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a973d1-79e6-4a03-bc35-8933b5398f2b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:23.113911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d8722-213f-4e93-8d5f-81004102774e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:23.114562 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d8722-213f-4e93-8d5f-81004102774e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:23.114840 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d8722-213f-4e93-8d5f-81004102774e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:23.115110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d8722-213f-4e93-8d5f-81004102774e None admin] PATH_INFO: `/v3/projects/b53a74f4aa094366a6811ccfa60d30f1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:23.116230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d8722-213f-4e93-8d5f-81004102774e None admin] RBAC: Authorizing `identity:get_project(project_id=b53a74f4aa094366a6811ccfa60d30f1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:23.117647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d8722-213f-4e93-8d5f-81004102774e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:23.119319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 605/1199] 10.222.0.62 () {56 vars in 1186 bytes} [Fri Sep 9 11:55:22 2022] GET /identity/v3/projects/b53a74f4aa094366a6811ccfa60d30f1 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:23.125467 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91be42d-d567-4dbe-a0c5-a858c99e9047 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:55:23.229212 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91be42d-d567-4dbe-a0c5-a858c99e9047 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:55:23.263024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a73dd3-b7a4-472b-8856-6d2d85635078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:55:23.263622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a73dd3-b7a4-472b-8856-6d2d85635078 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:55:23.263900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a73dd3-b7a4-472b-8856-6d2d85635078 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:55:23.264171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a73dd3-b7a4-472b-8856-6d2d85635078 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:55:23.264891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a73dd3-b7a4-472b-8856-6d2d85635078 None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=b53a74f4aa094366a6811ccfa60d30f1, user.id=5221d5c2baa3491e969a208f18d93438) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:55:23.265198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a73dd3-b7a4-472b-8856-6d2d85635078 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:55:23.266645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a73dd3-b7a4-472b-8856-6d2d85635078 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:55:23.272720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 595/1200] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:55:23 2022] GET /identity/v3/role_assignments?user.id=5221d5c2baa3491e969a208f18d93438&scope.project.id=b53a74f4aa094366a6811ccfa60d30f1&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:55:33.109761 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=42 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:55:33.270493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=72 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:19.524452 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b98269-ee46-4627-8db4-759941015531 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:19.525365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b98269-ee46-4627-8db4-759941015531 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:19.525537 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b98269-ee46-4627-8db4-759941015531 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:19.527969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 606/1201] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:56:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:56:19.539375 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84b37f-e0f4-45c4-960c-60b56c865ab8 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:19.544161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84b37f-e0f4-45c4-960c-60b56c865ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:19.544161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84b37f-e0f4-45c4-960c-60b56c865ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:19.562463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-2a84b37f-e0f4-45c4-960c-60b56c865ab8 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:19.563954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2a84b37f-e0f4-45c4-960c-60b56c865ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:19.673152 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 596/1202] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:56:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:56:19.972841 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b98269-ee46-4627-8db4-759941015531 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:20.105560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b98269-ee46-4627-8db4-759941015531 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:20.153320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaf711-1911-4b1d-b19f-9f40551e2cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:20.154179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaf711-1911-4b1d-b19f-9f40551e2cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:20.154403 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaf711-1911-4b1d-b19f-9f40551e2cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:20.154759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaf711-1911-4b1d-b19f-9f40551e2cb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:20.168107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeaf711-1911-4b1d-b19f-9f40551e2cb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:20.169955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeaf711-1911-4b1d-b19f-9f40551e2cb6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-adeaf711-1911-4b1d-b19f-9f40551e2cb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:56:20.194048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:20.195160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:56:20.196206 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:56:20.196206 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:20.196206 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:20.196206 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 607/1203] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:20.202931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84b37f-e0f4-45c4-960c-60b56c865ab8 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:20.355816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84b37f-e0f4-45c4-960c-60b56c865ab8 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:20.399613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:20.407806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4524e0a-34a1-46eb-a3bc-8d772af6c471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:20.408470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4524e0a-34a1-46eb-a3bc-8d772af6c471 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:20.408755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4524e0a-34a1-46eb-a3bc-8d772af6c471 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:20.409026 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4524e0a-34a1-46eb-a3bc-8d772af6c471 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:20.409739 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4524e0a-34a1-46eb-a3bc-8d772af6c471 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:20.410097 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4524e0a-34a1-46eb-a3bc-8d772af6c471 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:20.411632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4524e0a-34a1-46eb-a3bc-8d772af6c471 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:20.430292 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 597/1204] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:56:20.432545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaf711-1911-4b1d-b19f-9f40551e2cb6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:20.509355 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:20.583246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaf711-1911-4b1d-b19f-9f40551e2cb6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:20.649127 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c26c6-a74c-4d37-9dc5-5a2254e15799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:20.650210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c26c6-a74c-4d37-9dc5-5a2254e15799 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:20.650769 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c26c6-a74c-4d37-9dc5-5a2254e15799 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:20.651256 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c26c6-a74c-4d37-9dc5-5a2254e15799 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:20.652214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c26c6-a74c-4d37-9dc5-5a2254e15799 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:20.654187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c26c6-a74c-4d37-9dc5-5a2254e15799 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:20.690147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 608/1205] 10.222.0.62 () {60 vars in 1132 bytes} [Fri Sep 9 11:56:20 2022] POST /identity/v3/users => generated 251 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:56:23.004081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404902d8-58a3-49f9-93bb-47165910e71b None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:23.004647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404902d8-58a3-49f9-93bb-47165910e71b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:23.004647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404902d8-58a3-49f9-93bb-47165910e71b None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:23.006025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 598/1206] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:56:23.011553 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d9a2a-5a3f-4d31-b737-09da8701af92 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:23.012002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d9a2a-5a3f-4d31-b737-09da8701af92 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:23.012099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d9a2a-5a3f-4d31-b737-09da8701af92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:23.031290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2b8d9a2a-5a3f-4d31-b737-09da8701af92 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:23.032487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2b8d9a2a-5a3f-4d31-b737-09da8701af92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:23.105431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 609/1207] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:56:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:56:23.381377 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404902d8-58a3-49f9-93bb-47165910e71b None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:23.510062 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404902d8-58a3-49f9-93bb-47165910e71b None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:23.553360 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee622a5-768c-4e3e-9ed2-5fdc593788d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:23.553896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee622a5-768c-4e3e-9ed2-5fdc593788d1 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:23.554199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee622a5-768c-4e3e-9ed2-5fdc593788d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:23.554475 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee622a5-768c-4e3e-9ed2-5fdc593788d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:23.565942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee622a5-768c-4e3e-9ed2-5fdc593788d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-2ee622a5-768c-4e3e-9ed2-5fdc593788d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:56:23.570855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:23.571836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:23.572703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:23.572703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:23.572703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:23.572703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:23.572703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:56:23.572703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:56:23.572703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:56:23.572703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:56:23.572703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:56:23.572703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:56:23.572703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:56:23.572703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:23.577518 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 599/1208] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:56:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:23.578611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d9a2a-5a3f-4d31-b737-09da8701af92 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:23.622924 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:23.701435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d9a2a-5a3f-4d31-b737-09da8701af92 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:23.745882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72855809-8dd7-4aea-bcee-5dd452f94ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:23.746630 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72855809-8dd7-4aea-bcee-5dd452f94ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:23.746907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72855809-8dd7-4aea-bcee-5dd452f94ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:23.747174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72855809-8dd7-4aea-bcee-5dd452f94ad0 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:23.747838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72855809-8dd7-4aea-bcee-5dd452f94ad0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:23.748151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72855809-8dd7-4aea-bcee-5dd452f94ad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:23.749969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72855809-8dd7-4aea-bcee-5dd452f94ad0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:23.766193 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 610/1209] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:56:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:56:23.767133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee622a5-768c-4e3e-9ed2-5fdc593788d1 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:23.832295 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:23.905810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee622a5-768c-4e3e-9ed2-5fdc593788d1 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:23.945924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b2561-3600-4b69-906f-56b0f83f0278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:23.945924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b2561-3600-4b69-906f-56b0f83f0278 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:23.945924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b2561-3600-4b69-906f-56b0f83f0278 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:23.945924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b2561-3600-4b69-906f-56b0f83f0278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:23.959569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b2561-3600-4b69-906f-56b0f83f0278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:23.961259 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b2561-3600-4b69-906f-56b0f83f0278 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-1c4b2561-3600-4b69-906f-56b0f83f0278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:56:23.979220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:23.981045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:56:23.982733 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:56:23.982733 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:23.982733 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:23.982733 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 600/1210] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:23.988156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72855809-8dd7-4aea-bcee-5dd452f94ad0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:24.108679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:24.112126 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72855809-8dd7-4aea-bcee-5dd452f94ad0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:24.155945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621bc4e-0164-4210-b1f7-99a17a47835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:24.156563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621bc4e-0164-4210-b1f7-99a17a47835c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:24.157078 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621bc4e-0164-4210-b1f7-99a17a47835c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:24.157078 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621bc4e-0164-4210-b1f7-99a17a47835c None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:24.157721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621bc4e-0164-4210-b1f7-99a17a47835c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:24.158016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621bc4e-0164-4210-b1f7-99a17a47835c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:24.159403 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621bc4e-0164-4210-b1f7-99a17a47835c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:24.178260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 611/1211] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:24.179829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b2561-3600-4b69-906f-56b0f83f0278 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:24.299999 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b2561-3600-4b69-906f-56b0f83f0278 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:24.302054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:24.338916 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679e8b5-cb61-4434-b65d-095781763929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:24.339550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679e8b5-cb61-4434-b65d-095781763929 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:24.339832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679e8b5-cb61-4434-b65d-095781763929 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:24.340103 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679e8b5-cb61-4434-b65d-095781763929 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:24.349003 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9679e8b5-cb61-4434-b65d-095781763929 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:24.350478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9679e8b5-cb61-4434-b65d-095781763929 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-9679e8b5-cb61-4434-b65d-095781763929 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:56:24.363692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:24.364712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:24.365668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:24.365668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:24.365668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:56:24.365668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:56:24.365668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:56:24.365668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:56:24.365668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:56:24.365668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:56:24.365668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:56:24.365668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:24.365668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 601/1212] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:24.371357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621bc4e-0164-4210-b1f7-99a17a47835c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:24.485903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621bc4e-0164-4210-b1f7-99a17a47835c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:24.522530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f262777-68ce-4e30-ba6c-e0e232af9e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:24.523187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f262777-68ce-4e30-ba6c-e0e232af9e39 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:24.523399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f262777-68ce-4e30-ba6c-e0e232af9e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:24.523706 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f262777-68ce-4e30-ba6c-e0e232af9e39 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:24.524420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f262777-68ce-4e30-ba6c-e0e232af9e39 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:24.524689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f262777-68ce-4e30-ba6c-e0e232af9e39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:24.526144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f262777-68ce-4e30-ba6c-e0e232af9e39 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:24.550704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 612/1213] 10.222.0.62 () {56 vars in 1143 bytes} [Fri Sep 9 11:56:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:24.551964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679e8b5-cb61-4434-b65d-095781763929 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:24.676465 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679e8b5-cb61-4434-b65d-095781763929 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:24.715787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e81b45-95ca-4dda-ba6b-7ad5ebc6856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:24.716395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e81b45-95ca-4dda-ba6b-7ad5ebc6856e None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:24.716669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e81b45-95ca-4dda-ba6b-7ad5ebc6856e None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:24.716932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e81b45-95ca-4dda-ba6b-7ad5ebc6856e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:24.755634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:24.801706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e81b45-95ca-4dda-ba6b-7ad5ebc6856e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:24.803329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e81b45-95ca-4dda-ba6b-7ad5ebc6856e None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:24.922919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 602/1214] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:56:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:56:24.928975 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f262777-68ce-4e30-ba6c-e0e232af9e39 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:24.954578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:25.051291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f262777-68ce-4e30-ba6c-e0e232af9e39 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:25.093309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4865e39-0eef-4564-b00f-faa2191bf250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:25.095087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4865e39-0eef-4564-b00f-faa2191bf250 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:25.095087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4865e39-0eef-4564-b00f-faa2191bf250 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:25.095087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4865e39-0eef-4564-b00f-faa2191bf250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:25.106294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4865e39-0eef-4564-b00f-faa2191bf250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:25.108405 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4865e39-0eef-4564-b00f-faa2191bf250 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-f4865e39-0eef-4564-b00f-faa2191bf250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:56:25.134447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:25.136112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:56:25.137027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:56:25.137027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:25.137027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:25.137027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 613/1215] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:25.144292 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e81b45-95ca-4dda-ba6b-7ad5ebc6856e None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:25.231470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:25.282451 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e81b45-95ca-4dda-ba6b-7ad5ebc6856e None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:25.321971 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e961a2-1b57-44b8-bd10-afffe99e192a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:25.322529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e961a2-1b57-44b8-bd10-afffe99e192a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:25.322807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e961a2-1b57-44b8-bd10-afffe99e192a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:25.323074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e961a2-1b57-44b8-bd10-afffe99e192a None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:25.323722 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e961a2-1b57-44b8-bd10-afffe99e192a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:25.324019 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e961a2-1b57-44b8-bd10-afffe99e192a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:25.325365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e961a2-1b57-44b8-bd10-afffe99e192a None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:25.337502 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 603/1216] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:25.344064 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4865e39-0eef-4564-b00f-faa2191bf250 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:25.425251 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:25.461574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4865e39-0eef-4564-b00f-faa2191bf250 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:25.500481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6aa364-4aa7-42cf-9223-d3c061a05e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:25.501134 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6aa364-4aa7-42cf-9223-d3c061a05e4b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:25.501412 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6aa364-4aa7-42cf-9223-d3c061a05e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:25.501713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6aa364-4aa7-42cf-9223-d3c061a05e4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:25.508700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6aa364-4aa7-42cf-9223-d3c061a05e4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:25.510395 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6aa364-4aa7-42cf-9223-d3c061a05e4b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:25.525319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-8c6aa364-4aa7-42cf-9223-d3c061a05e4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:56:25.525319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:25.525319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:25.525319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:25.525319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:25.525319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:25.525319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:25.525319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:25.525319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:25.525319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:25.525319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:25.525319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:25.525319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:56:25.525319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:56:25.525319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:25.526569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:25.527677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:56:25.527677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:56:25.527677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:56:25.527677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:56:25.527677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:56:25.527677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:56:25.527677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:56:25.527677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:25.527677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 614/1217] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:56:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:25.534290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e961a2-1b57-44b8-bd10-afffe99e192a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:25.652094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e961a2-1b57-44b8-bd10-afffe99e192a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:25.676929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:25.691157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5561e55-97a7-4607-8056-4399d1c72d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:25.691801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5561e55-97a7-4607-8056-4399d1c72d66 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:25.692112 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5561e55-97a7-4607-8056-4399d1c72d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:25.692432 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5561e55-97a7-4607-8056-4399d1c72d66 None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:25.693237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5561e55-97a7-4607-8056-4399d1c72d66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:25.693595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5561e55-97a7-4607-8056-4399d1c72d66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:25.695220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5561e55-97a7-4607-8056-4399d1c72d66 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:25.711604 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 604/1218] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:56:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:25.718458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6aa364-4aa7-42cf-9223-d3c061a05e4b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:25.833028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6aa364-4aa7-42cf-9223-d3c061a05e4b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:25.867123 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:25.876966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40aadf-47fd-418a-9ac8-1eb91972bd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:25.877802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40aadf-47fd-418a-9ac8-1eb91972bd3d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:25.878188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40aadf-47fd-418a-9ac8-1eb91972bd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:25.878511 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40aadf-47fd-418a-9ac8-1eb91972bd3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:25.879401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40aadf-47fd-418a-9ac8-1eb91972bd3d None admin] RBAC: Adding query filter params (role.id=035f7d9f445d406eb2e83ca90fc4d5ca, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=c58bae3ef7784671ae247f4ad4199c0a) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:25.879713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40aadf-47fd-418a-9ac8-1eb91972bd3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:25.881183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40aadf-47fd-418a-9ac8-1eb91972bd3d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:25.890862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 615/1219] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:56:25 2022] GET /identity/v3/role_assignments?user.id=c58bae3ef7784671ae247f4ad4199c0a&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=035f7d9f445d406eb2e83ca90fc4d5ca => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:28.106590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8143c-fafc-48e9-adb6-0719a185e749 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:28.107427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8143c-fafc-48e9-adb6-0719a185e749 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:28.107427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8143c-fafc-48e9-adb6-0719a185e749 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:28.108770 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 605/1220] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:56:28.116166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9d53a-e405-4410-976b-30019b89f3da None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:28.116807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9d53a-e405-4410-976b-30019b89f3da None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:28.116914 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9d53a-e405-4410-976b-30019b89f3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:28.141646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-85c9d53a-e405-4410-976b-30019b89f3da None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:28.143303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-85c9d53a-e405-4410-976b-30019b89f3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:28.228134 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 616/1221] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:56:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:56:28.523335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8143c-fafc-48e9-adb6-0719a185e749 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:28.646162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8143c-fafc-48e9-adb6-0719a185e749 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:28.684791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b437c24-b845-49bc-b7d8-0306bb0cf81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:28.685427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b437c24-b845-49bc-b7d8-0306bb0cf81a None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:28.685707 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b437c24-b845-49bc-b7d8-0306bb0cf81a None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:28.686638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b437c24-b845-49bc-b7d8-0306bb0cf81a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:28.696911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b437c24-b845-49bc-b7d8-0306bb0cf81a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-8b437c24-b845-49bc-b7d8-0306bb0cf81a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:56:28.702412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:28.703658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:28.704584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:28.704584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:28.704584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:28.704584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:28.704584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:56:28.704584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:56:28.704584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:56:28.704584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:56:28.704584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:56:28.704584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:56:28.704584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:56:28.704584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:28.704584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 606/1222] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:56:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:56:28.710547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9d53a-e405-4410-976b-30019b89f3da None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:28.825098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9d53a-e405-4410-976b-30019b89f3da None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:28.852299 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:28.865261 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35196b2b-7701-444c-8d50-6b47ab4286c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:28.866021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35196b2b-7701-444c-8d50-6b47ab4286c9 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:28.866380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35196b2b-7701-444c-8d50-6b47ab4286c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:28.866659 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35196b2b-7701-444c-8d50-6b47ab4286c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:28.867372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35196b2b-7701-444c-8d50-6b47ab4286c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:28.867694 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35196b2b-7701-444c-8d50-6b47ab4286c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:28.869235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35196b2b-7701-444c-8d50-6b47ab4286c9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:28.879331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 617/1223] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:56:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:28.885242 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b437c24-b845-49bc-b7d8-0306bb0cf81a None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:28.937968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:29.003464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b437c24-b845-49bc-b7d8-0306bb0cf81a None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:29.039264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804a85c-b5a8-4071-af74-d827a8049a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:29.039896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804a85c-b5a8-4071-af74-d827a8049a28 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:29.040174 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804a85c-b5a8-4071-af74-d827a8049a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:29.040443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804a85c-b5a8-4071-af74-d827a8049a28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:29.052325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804a85c-b5a8-4071-af74-d827a8049a28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:29.053798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804a85c-b5a8-4071-af74-d827a8049a28 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-5804a85c-b5a8-4071-af74-d827a8049a28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:56:29.069731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:29.071189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:56:29.072402 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:56:29.072402 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:29.072402 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:29.075900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 607/1224] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:29.077049 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35196b2b-7701-444c-8d50-6b47ab4286c9 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:29.195935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35196b2b-7701-444c-8d50-6b47ab4286c9 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:29.241714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a3eb2-4493-41ed-833c-7047f67d5344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:29.242505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a3eb2-4493-41ed-833c-7047f67d5344 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:29.242790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a3eb2-4493-41ed-833c-7047f67d5344 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:29.243078 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a3eb2-4493-41ed-833c-7047f67d5344 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:29.243846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a3eb2-4493-41ed-833c-7047f67d5344 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:29.244276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a3eb2-4493-41ed-833c-7047f67d5344 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:29.246087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a3eb2-4493-41ed-833c-7047f67d5344 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:29.261476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 618/1225] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:29.267698 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804a85c-b5a8-4071-af74-d827a8049a28 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:29.380044 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:29.388190 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804a85c-b5a8-4071-af74-d827a8049a28 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:29.425685 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c211970-6679-428f-8a76-1c0cf80ca6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:29.426346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c211970-6679-428f-8a76-1c0cf80ca6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:29.426977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c211970-6679-428f-8a76-1c0cf80ca6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:29.426977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c211970-6679-428f-8a76-1c0cf80ca6e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:29.434995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c211970-6679-428f-8a76-1c0cf80ca6e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:29.437789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c211970-6679-428f-8a76-1c0cf80ca6e4 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-1c211970-6679-428f-8a76-1c0cf80ca6e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:56:29.452011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:29.454335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:29.456575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:29.456575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:29.456575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:56:29.456575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:56:29.456575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:56:29.456575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:56:29.456575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:56:29.456575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:56:29.456575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:56:29.456575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:29.456575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 608/1226] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:29.459570 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a3eb2-4493-41ed-833c-7047f67d5344 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:29.515749 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:29.577246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a3eb2-4493-41ed-833c-7047f67d5344 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:29.626347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56719a-d6d2-4be8-b613-b988567677a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:29.627282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56719a-d6d2-4be8-b613-b988567677a0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:29.627676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56719a-d6d2-4be8-b613-b988567677a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:29.628055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f56719a-d6d2-4be8-b613-b988567677a0 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:29.629006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f56719a-d6d2-4be8-b613-b988567677a0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:29.629275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f56719a-d6d2-4be8-b613-b988567677a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:29.630855 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f56719a-d6d2-4be8-b613-b988567677a0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:29.652583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 619/1227] 10.222.0.62 () {56 vars in 1143 bytes} [Fri Sep 9 11:56:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:29.658426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c211970-6679-428f-8a76-1c0cf80ca6e4 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:29.775124 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c211970-6679-428f-8a76-1c0cf80ca6e4 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:29.815180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce552ba0-5dd8-425a-95c4-1640346c84b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:29.815929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce552ba0-5dd8-425a-95c4-1640346c84b0 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:29.816268 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce552ba0-5dd8-425a-95c4-1640346c84b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:29.816603 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce552ba0-5dd8-425a-95c4-1640346c84b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:29.828265 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce552ba0-5dd8-425a-95c4-1640346c84b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:29.829771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce552ba0-5dd8-425a-95c4-1640346c84b0 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-ce552ba0-5dd8-425a-95c4-1640346c84b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:56:29.844215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:29.845442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:56:29.846666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:56:29.846666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:29.846666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:29.846666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 609/1228] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:29.851809 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56719a-d6d2-4be8-b613-b988567677a0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:29.989324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f56719a-d6d2-4be8-b613-b988567677a0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:29.991910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:30.030597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bb1e8-3f1f-4028-b2b5-06a8ee00e2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:30.031548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bb1e8-3f1f-4028-b2b5-06a8ee00e2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:30.031955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bb1e8-3f1f-4028-b2b5-06a8ee00e2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:30.032361 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bb1e8-3f1f-4028-b2b5-06a8ee00e2b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:30.033245 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bb1e8-3f1f-4028-b2b5-06a8ee00e2b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:30.033632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bb1e8-3f1f-4028-b2b5-06a8ee00e2b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:30.035381 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bb1e8-3f1f-4028-b2b5-06a8ee00e2b9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:30.056952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 620/1229] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:30.058538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce552ba0-5dd8-425a-95c4-1640346c84b0 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:30.069641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:30.174874 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce552ba0-5dd8-425a-95c4-1640346c84b0 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:30.210251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54937e06-46de-4f6e-938c-cba346540169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:30.210884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54937e06-46de-4f6e-938c-cba346540169 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:30.211180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54937e06-46de-4f6e-938c-cba346540169 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:30.211428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54937e06-46de-4f6e-938c-cba346540169 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:30.218814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54937e06-46de-4f6e-938c-cba346540169 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:30.220377 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54937e06-46de-4f6e-938c-cba346540169 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:30.236840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-54937e06-46de-4f6e-938c-cba346540169 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:56:30.236840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:30.236840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:30.236840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:30.236840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:30.236840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:30.236840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:30.236840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:30.236840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:30.236840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:30.236840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:30.236840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:30.236840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:56:30.236840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:56:30.236840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:30.238109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:30.239873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:56:30.239873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:56:30.239873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:56:30.239873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:56:30.239873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:56:30.239873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:56:30.239873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:56:30.239873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:30.239873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 610/1230] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:56:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:30.244466 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bb1e8-3f1f-4028-b2b5-06a8ee00e2b9 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:30.359240 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bb1e8-3f1f-4028-b2b5-06a8ee00e2b9 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:30.394720 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee60c5-8bbc-42c5-9349-1148754acc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:30.395455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee60c5-8bbc-42c5-9349-1148754acc61 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:30.395740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee60c5-8bbc-42c5-9349-1148754acc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:30.396038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee60c5-8bbc-42c5-9349-1148754acc61 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:30.396797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee60c5-8bbc-42c5-9349-1148754acc61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:30.397137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee60c5-8bbc-42c5-9349-1148754acc61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:30.398621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee60c5-8bbc-42c5-9349-1148754acc61 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:30.411505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 621/1231] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:56:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:30.417617 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54937e06-46de-4f6e-938c-cba346540169 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:30.499998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:30.540358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54937e06-46de-4f6e-938c-cba346540169 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:30.579395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc048d-843e-4299-a7e2-457e3a657356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:30.580200 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc048d-843e-4299-a7e2-457e3a657356 None admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:30.580555 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc048d-843e-4299-a7e2-457e3a657356 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:30.580856 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc048d-843e-4299-a7e2-457e3a657356 None admin] PATH_INFO: `/v3/projects/32b06df0c5504592b012585f193386e7/users/c58bae3ef7784671ae247f4ad4199c0a/roles/035f7d9f445d406eb2e83ca90fc4d5ca` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:30.601623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc048d-843e-4299-a7e2-457e3a657356 None admin] RBAC: Authorizing `identity:create_grant(project_id=32b06df0c5504592b012585f193386e7, user_id=c58bae3ef7784671ae247f4ad4199c0a, role_id=035f7d9f445d406eb2e83ca90fc4d5ca)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:30.603397 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc048d-843e-4299-a7e2-457e3a657356 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:30.620258 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 611/1232] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:56:30 2022] PUT /identity/v3/projects/32b06df0c5504592b012585f193386e7/users/c58bae3ef7784671ae247f4ad4199c0a/roles/035f7d9f445d406eb2e83ca90fc4d5ca => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:32.940755 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bff9e7-2347-4424-bd79-30fc7e94d8a7 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:32.941320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bff9e7-2347-4424-bd79-30fc7e94d8a7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:32.941320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bff9e7-2347-4424-bd79-30fc7e94d8a7 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:32.942703 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 622/1233] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:56:32.949236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f87db5-28ef-444f-98e4-824f9794b513 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:32.949769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f87db5-28ef-444f-98e4-824f9794b513 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:32.949936 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f87db5-28ef-444f-98e4-824f9794b513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:32.972323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-69f87db5-28ef-444f-98e4-824f9794b513 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:32.973759 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-69f87db5-28ef-444f-98e4-824f9794b513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:33.051925 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 612/1234] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:56:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:56:33.351600 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bff9e7-2347-4424-bd79-30fc7e94d8a7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:33.381658 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:33.472737 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bff9e7-2347-4424-bd79-30fc7e94d8a7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:33.511771 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a459f7-959c-48d5-b239-fc5974d52543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:33.512410 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a459f7-959c-48d5-b239-fc5974d52543 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:33.512727 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a459f7-959c-48d5-b239-fc5974d52543 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:33.513033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a459f7-959c-48d5-b239-fc5974d52543 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:33.525621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a459f7-959c-48d5-b239-fc5974d52543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-f1a459f7-959c-48d5-b239-fc5974d52543 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:56:33.531469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:33.532642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:33.533728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:33.533728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:33.533728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:33.533728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:33.533728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:56:33.533728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:56:33.533728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:56:33.533728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:56:33.533728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:56:33.533728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:56:33.533728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:56:33.533728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:33.533728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 623/1235] 10.222.0.62 () {56 vars in 1102 bytes} [Fri Sep 9 11:56:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:33.538803 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f87db5-28ef-444f-98e4-824f9794b513 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:33.596465 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:33.672337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f87db5-28ef-444f-98e4-824f9794b513 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:33.712461 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe466a-70d0-4d76-95b3-e2dc34064be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:33.713072 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe466a-70d0-4d76-95b3-e2dc34064be7 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:33.713347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe466a-70d0-4d76-95b3-e2dc34064be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:33.713615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe466a-70d0-4d76-95b3-e2dc34064be7 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:33.714369 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe466a-70d0-4d76-95b3-e2dc34064be7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:33.714669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe466a-70d0-4d76-95b3-e2dc34064be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:33.716037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe466a-70d0-4d76-95b3-e2dc34064be7 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:33.724986 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 613/1236] 10.222.0.62 () {56 vars in 1103 bytes} [Fri Sep 9 11:56:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:33.731332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a459f7-959c-48d5-b239-fc5974d52543 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:33.821357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:33.839609 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a459f7-959c-48d5-b239-fc5974d52543 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:33.877591 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4afa4a-0b79-48cd-9b4d-0e7839e8ef93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:33.878263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4afa4a-0b79-48cd-9b4d-0e7839e8ef93 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:33.878546 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4afa4a-0b79-48cd-9b4d-0e7839e8ef93 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:33.878819 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4afa4a-0b79-48cd-9b4d-0e7839e8ef93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:33.893396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4afa4a-0b79-48cd-9b4d-0e7839e8ef93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:33.896119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4afa4a-0b79-48cd-9b4d-0e7839e8ef93 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-9f4afa4a-0b79-48cd-9b4d-0e7839e8ef93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:56:33.918696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:33.919959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:56:33.921030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:56:33.921030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:33.921030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:33.921030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 624/1237] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:33.926397 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe466a-70d0-4d76-95b3-e2dc34064be7 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:34.053168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe466a-70d0-4d76-95b3-e2dc34064be7 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:34.065322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:34.095674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458246c-cc61-4824-a19f-924ce21dfc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:34.096327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458246c-cc61-4824-a19f-924ce21dfc5c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:34.096627 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458246c-cc61-4824-a19f-924ce21dfc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:34.096939 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458246c-cc61-4824-a19f-924ce21dfc5c None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:34.097630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c458246c-cc61-4824-a19f-924ce21dfc5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:34.097972 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c458246c-cc61-4824-a19f-924ce21dfc5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:34.099476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c458246c-cc61-4824-a19f-924ce21dfc5c None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:34.113921 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 614/1238] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:34.120384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4afa4a-0b79-48cd-9b4d-0e7839e8ef93 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:34.231793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4afa4a-0b79-48cd-9b4d-0e7839e8ef93 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:34.267911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e0b2a-0fc4-4a95-9b0e-f4ef06c1eb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:34.268556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e0b2a-0fc4-4a95-9b0e-f4ef06c1eb09 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:34.268831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e0b2a-0fc4-4a95-9b0e-f4ef06c1eb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:34.269133 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e0b2a-0fc4-4a95-9b0e-f4ef06c1eb09 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:34.272125 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:34.277488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e0b2a-0fc4-4a95-9b0e-f4ef06c1eb09 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:34.279152 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e0b2a-0fc4-4a95-9b0e-f4ef06c1eb09 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-3d5e0b2a-0fc4-4a95-9b0e-f4ef06c1eb09 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:56:34.292648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:34.293715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:34.295270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:34.295270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:34.295270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:56:34.295270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:56:34.295270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:56:34.295270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:56:34.295270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:56:34.295270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:56:34.295270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:56:34.295270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:34.295270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 625/1239] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:56:34.300194 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458246c-cc61-4824-a19f-924ce21dfc5c None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:34.423469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458246c-cc61-4824-a19f-924ce21dfc5c None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:34.461089 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521d70c-ce71-4000-935e-3f51a78becf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:34.461727 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521d70c-ce71-4000-935e-3f51a78becf0 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:34.462039 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521d70c-ce71-4000-935e-3f51a78becf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:34.462341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521d70c-ce71-4000-935e-3f51a78becf0 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:34.463112 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521d70c-ce71-4000-935e-3f51a78becf0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:34.463543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521d70c-ce71-4000-935e-3f51a78becf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:34.465188 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521d70c-ce71-4000-935e-3f51a78becf0 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:34.484908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 615/1240] 10.222.0.62 () {56 vars in 1143 bytes} [Fri Sep 9 11:56:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:34.491436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e0b2a-0fc4-4a95-9b0e-f4ef06c1eb09 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:34.607438 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e0b2a-0fc4-4a95-9b0e-f4ef06c1eb09 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:34.656593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18d192-2cf1-4e7e-821c-2924dcdcdbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:34.657492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18d192-2cf1-4e7e-821c-2924dcdcdbd8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:34.657747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18d192-2cf1-4e7e-821c-2924dcdcdbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:34.658020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18d192-2cf1-4e7e-821c-2924dcdcdbd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:34.721834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:34.729206 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18d192-2cf1-4e7e-821c-2924dcdcdbd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:34.730672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18d192-2cf1-4e7e-821c-2924dcdcdbd8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:34.846193 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 626/1241] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:56:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:56:34.848491 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521d70c-ce71-4000-935e-3f51a78becf0 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:34.883597 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:34.971272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521d70c-ce71-4000-935e-3f51a78becf0 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:35.023438 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af699d-5212-4fa0-9012-31da35fca24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:35.024250 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af699d-5212-4fa0-9012-31da35fca24c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:35.024721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af699d-5212-4fa0-9012-31da35fca24c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:35.025202 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af699d-5212-4fa0-9012-31da35fca24c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:35.041328 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af699d-5212-4fa0-9012-31da35fca24c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:35.043306 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af699d-5212-4fa0-9012-31da35fca24c None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-60af699d-5212-4fa0-9012-31da35fca24c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:56:35.064721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:35.067506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:56:35.070468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:56:35.070468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:35.070468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:35.070468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 616/1242] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:35.072937 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18d192-2cf1-4e7e-821c-2924dcdcdbd8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:35.199553 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:35.207477 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18d192-2cf1-4e7e-821c-2924dcdcdbd8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:35.247792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13506ed7-0aa4-4a84-85e5-946ea294fa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:35.248969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13506ed7-0aa4-4a84-85e5-946ea294fa28 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:35.249247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13506ed7-0aa4-4a84-85e5-946ea294fa28 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:35.249515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13506ed7-0aa4-4a84-85e5-946ea294fa28 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:35.250188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13506ed7-0aa4-4a84-85e5-946ea294fa28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:35.250493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13506ed7-0aa4-4a84-85e5-946ea294fa28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:35.251906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13506ed7-0aa4-4a84-85e5-946ea294fa28 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:35.266608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 627/1243] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:35.272569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af699d-5212-4fa0-9012-31da35fca24c None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:35.361609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:35.386885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af699d-5212-4fa0-9012-31da35fca24c None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:35.424939 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9b45a-d0a5-4ab3-97e9-cf8729d558a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:35.425636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9b45a-d0a5-4ab3-97e9-cf8729d558a3 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:35.425933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9b45a-d0a5-4ab3-97e9-cf8729d558a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:35.426256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9b45a-d0a5-4ab3-97e9-cf8729d558a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:35.433523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9b45a-d0a5-4ab3-97e9-cf8729d558a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:35.435162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9b45a-d0a5-4ab3-97e9-cf8729d558a3 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:35.448117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-30b9b45a-d0a5-4ab3-97e9-cf8729d558a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:56:35.448117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:35.448117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:35.448117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:35.448117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:35.448117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:35.448117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:35.448117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:35.448117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:35.448117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:35.448117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:35.448117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:35.448117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:56:35.448117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:56:35.448117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:35.449260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:35.450387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:56:35.450387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:56:35.450387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:56:35.450387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:56:35.450387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:56:35.450387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:56:35.450387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:56:35.450387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:35.450387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 617/1244] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:56:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:56:35.455818 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13506ed7-0aa4-4a84-85e5-946ea294fa28 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:35.570523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13506ed7-0aa4-4a84-85e5-946ea294fa28 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:35.613467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dae8d-5592-476f-a8d6-6a3390228823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:35.614014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dae8d-5592-476f-a8d6-6a3390228823 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:35.614459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dae8d-5592-476f-a8d6-6a3390228823 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:35.614640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dae8d-5592-476f-a8d6-6a3390228823 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:35.615364 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dae8d-5592-476f-a8d6-6a3390228823 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:35.615708 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dae8d-5592-476f-a8d6-6a3390228823 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:35.617271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dae8d-5592-476f-a8d6-6a3390228823 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:35.631192 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 628/1245] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:56:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:35.637601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9b45a-d0a5-4ab3-97e9-cf8729d558a3 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:35.748913 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9b45a-d0a5-4ab3-97e9-cf8729d558a3 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:35.787740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77633abd-4df7-4158-9f6b-1e14ddd01a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:35.788384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77633abd-4df7-4158-9f6b-1e14ddd01a01 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:35.788663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77633abd-4df7-4158-9f6b-1e14ddd01a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:35.788931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77633abd-4df7-4158-9f6b-1e14ddd01a01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:35.789664 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77633abd-4df7-4158-9f6b-1e14ddd01a01 None admin] RBAC: Adding query filter params (role.id=035f7d9f445d406eb2e83ca90fc4d5ca, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=c58bae3ef7784671ae247f4ad4199c0a) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:35.790013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77633abd-4df7-4158-9f6b-1e14ddd01a01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:35.791642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77633abd-4df7-4158-9f6b-1e14ddd01a01 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:35.798127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 618/1246] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:56:35 2022] GET /identity/v3/role_assignments?user.id=c58bae3ef7784671ae247f4ad4199c0a&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=035f7d9f445d406eb2e83ca90fc4d5ca => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:38.325411 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7483fd-7491-4168-b7fe-fddac0898bca None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:38.326012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7483fd-7491-4168-b7fe-fddac0898bca None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:38.326012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7483fd-7491-4168-b7fe-fddac0898bca None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:38.328825 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 629/1247] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:56:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:56:38.334185 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196bf34-f9ef-4727-9a3a-87b2f86fc6bb None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:38.334710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196bf34-f9ef-4727-9a3a-87b2f86fc6bb None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:38.334710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196bf34-f9ef-4727-9a3a-87b2f86fc6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:38.341832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:38.356373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a196bf34-f9ef-4727-9a3a-87b2f86fc6bb None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:38.358407 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a196bf34-f9ef-4727-9a3a-87b2f86fc6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:38.432508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 619/1248] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:56:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:56:38.715183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7483fd-7491-4168-b7fe-fddac0898bca None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:38.735578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:38.827570 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7483fd-7491-4168-b7fe-fddac0898bca None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:38.865259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628d625-64bc-438f-8b08-85e5e2ace944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:38.865890 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628d625-64bc-438f-8b08-85e5e2ace944 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:38.866238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628d625-64bc-438f-8b08-85e5e2ace944 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:38.866511 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628d625-64bc-438f-8b08-85e5e2ace944 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:38.877953 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628d625-64bc-438f-8b08-85e5e2ace944 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-0628d625-64bc-438f-8b08-85e5e2ace944 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:56:38.882839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:38.884267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:38.885344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:38.885344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:38.885344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:38.885344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:38.885344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:56:38.885344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:56:38.885344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:56:38.885344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:56:38.885344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:56:38.885344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:56:38.885344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:56:38.885344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:38.885344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 630/1249] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:56:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:56:38.890177 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196bf34-f9ef-4727-9a3a-87b2f86fc6bb None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:38.999887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196bf34-f9ef-4727-9a3a-87b2f86fc6bb None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:39.033264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:39.039824 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836720ad-74e9-4ecd-9e4d-d8fcc8b34bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:39.040630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836720ad-74e9-4ecd-9e4d-d8fcc8b34bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:39.040911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836720ad-74e9-4ecd-9e4d-d8fcc8b34bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:39.041222 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836720ad-74e9-4ecd-9e4d-d8fcc8b34bb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:39.042151 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836720ad-74e9-4ecd-9e4d-d8fcc8b34bb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:39.042691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836720ad-74e9-4ecd-9e4d-d8fcc8b34bb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:39.044376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836720ad-74e9-4ecd-9e4d-d8fcc8b34bb2 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:39.059359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 620/1250] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:56:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:39.060744 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628d625-64bc-438f-8b08-85e5e2ace944 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:39.146630 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:39.174175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628d625-64bc-438f-8b08-85e5e2ace944 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:39.213511 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e556019-851f-4c51-89ba-d0672a0322e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:39.214182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e556019-851f-4c51-89ba-d0672a0322e8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:39.214494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e556019-851f-4c51-89ba-d0672a0322e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:39.214818 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e556019-851f-4c51-89ba-d0672a0322e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:39.229170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e556019-851f-4c51-89ba-d0672a0322e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:39.230768 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e556019-851f-4c51-89ba-d0672a0322e8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-5e556019-851f-4c51-89ba-d0672a0322e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:56:39.247502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:39.249157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:56:39.250143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:56:39.250143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:39.250143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:39.250143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 631/1251] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:39.255705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836720ad-74e9-4ecd-9e4d-d8fcc8b34bb2 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:39.370236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836720ad-74e9-4ecd-9e4d-d8fcc8b34bb2 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:39.407930 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c1405-7556-4ef0-b7d7-6fa2abe23d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:39.408554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c1405-7556-4ef0-b7d7-6fa2abe23d2b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:39.408832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c1405-7556-4ef0-b7d7-6fa2abe23d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:39.409104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c1405-7556-4ef0-b7d7-6fa2abe23d2b None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:39.409794 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c1405-7556-4ef0-b7d7-6fa2abe23d2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:39.410100 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c1405-7556-4ef0-b7d7-6fa2abe23d2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:39.411525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c1405-7556-4ef0-b7d7-6fa2abe23d2b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:39.429696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 621/1252] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:39.430920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e556019-851f-4c51-89ba-d0672a0322e8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:39.538580 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e556019-851f-4c51-89ba-d0672a0322e8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:39.566649 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:39.575051 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b17d42-d3a5-4f59-8e17-d4d7c08726dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:39.575701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b17d42-d3a5-4f59-8e17-d4d7c08726dd None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:39.575975 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b17d42-d3a5-4f59-8e17-d4d7c08726dd None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:39.576338 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b17d42-d3a5-4f59-8e17-d4d7c08726dd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:39.586143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b17d42-d3a5-4f59-8e17-d4d7c08726dd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:39.587858 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b17d42-d3a5-4f59-8e17-d4d7c08726dd None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-63b17d42-d3a5-4f59-8e17-d4d7c08726dd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:56:39.606893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:39.608656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:39.609868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:39.609868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:39.609868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:56:39.609868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:56:39.609868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:56:39.609868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:56:39.609868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:56:39.609868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:56:39.609868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:56:39.609868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:39.614801 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 632/1253] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:39.616263 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c1405-7556-4ef0-b7d7-6fa2abe23d2b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:39.684916 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:39.731986 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c1405-7556-4ef0-b7d7-6fa2abe23d2b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:39.770464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b5f9b-7b56-4d64-97cb-58f9536ac8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:39.771097 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b5f9b-7b56-4d64-97cb-58f9536ac8fa None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:39.771343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b5f9b-7b56-4d64-97cb-58f9536ac8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:39.771614 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b5f9b-7b56-4d64-97cb-58f9536ac8fa None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:39.772353 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b5f9b-7b56-4d64-97cb-58f9536ac8fa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:39.772650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b5f9b-7b56-4d64-97cb-58f9536ac8fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:39.774329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b5f9b-7b56-4d64-97cb-58f9536ac8fa None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:39.793483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 622/1254] 10.222.0.62 () {56 vars in 1143 bytes} [Fri Sep 9 11:56:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:39.799963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b17d42-d3a5-4f59-8e17-d4d7c08726dd None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:39.901162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b17d42-d3a5-4f59-8e17-d4d7c08726dd None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:39.935501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafc5be-bc86-4a98-912a-9ae6faf0d36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:39.936107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafc5be-bc86-4a98-912a-9ae6faf0d36e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:39.936384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafc5be-bc86-4a98-912a-9ae6faf0d36e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:39.936653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafc5be-bc86-4a98-912a-9ae6faf0d36e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:39.995765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:40.008455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bafc5be-bc86-4a98-912a-9ae6faf0d36e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:40.009907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bafc5be-bc86-4a98-912a-9ae6faf0d36e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:40.105150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 633/1255] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:56:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:56:40.111898 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b5f9b-7b56-4d64-97cb-58f9536ac8fa None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:40.211769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b5f9b-7b56-4d64-97cb-58f9536ac8fa None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:40.225822 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:40.246911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71ca43-0b88-4a52-965c-139c8f72633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:40.247531 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71ca43-0b88-4a52-965c-139c8f72633e None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:40.247851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71ca43-0b88-4a52-965c-139c8f72633e None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:40.248117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71ca43-0b88-4a52-965c-139c8f72633e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:40.257712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e71ca43-0b88-4a52-965c-139c8f72633e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:40.259160 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e71ca43-0b88-4a52-965c-139c8f72633e None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-4e71ca43-0b88-4a52-965c-139c8f72633e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:56:40.273743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:40.274901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:56:40.276162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:56:40.276162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:40.276162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:40.276162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 623/1256] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:40.281330 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafc5be-bc86-4a98-912a-9ae6faf0d36e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:40.368853 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:40.381292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafc5be-bc86-4a98-912a-9ae6faf0d36e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:40.415212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373325e-2c17-42f4-baf4-0e18128715db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:40.415812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373325e-2c17-42f4-baf4-0e18128715db None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:40.416093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373325e-2c17-42f4-baf4-0e18128715db None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:40.416369 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373325e-2c17-42f4-baf4-0e18128715db None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:40.417003 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373325e-2c17-42f4-baf4-0e18128715db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:40.417303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373325e-2c17-42f4-baf4-0e18128715db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:40.418688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373325e-2c17-42f4-baf4-0e18128715db None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:40.429620 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 634/1257] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:40.435315 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71ca43-0b88-4a52-965c-139c8f72633e None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:40.533626 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71ca43-0b88-4a52-965c-139c8f72633e None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:40.566609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a6411-8cd7-427c-9492-eb7c010e2a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:40.567220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a6411-8cd7-427c-9492-eb7c010e2a85 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:40.567498 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a6411-8cd7-427c-9492-eb7c010e2a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:40.567769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a6411-8cd7-427c-9492-eb7c010e2a85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:40.574361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a6411-8cd7-427c-9492-eb7c010e2a85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:40.575890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a6411-8cd7-427c-9492-eb7c010e2a85 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:40.589957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-d04a6411-8cd7-427c-9492-eb7c010e2a85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:56:40.589957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:40.589957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:40.589957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:40.589957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:40.589957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:40.589957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:40.589957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:40.589957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:40.589957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:40.589957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:40.589957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:40.589957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:56:40.589957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:56:40.589957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:40.591056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:40.592020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:56:40.592020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:56:40.592020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:56:40.592020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:56:40.592020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:56:40.592020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:56:40.592020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:56:40.592020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:40.592020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 624/1258] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:56:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:40.598032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373325e-2c17-42f4-baf4-0e18128715db None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:40.703515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373325e-2c17-42f4-baf4-0e18128715db None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:40.735300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e0483-21da-467d-8dbf-f109fb4b846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:40.735900 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e0483-21da-467d-8dbf-f109fb4b846d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:40.736174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e0483-21da-467d-8dbf-f109fb4b846d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:40.736441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e0483-21da-467d-8dbf-f109fb4b846d None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:40.737132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e0483-21da-467d-8dbf-f109fb4b846d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:40.737431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e0483-21da-467d-8dbf-f109fb4b846d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:40.738867 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e0483-21da-467d-8dbf-f109fb4b846d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:40.750890 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 635/1259] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:56:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:40.756974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a6411-8cd7-427c-9492-eb7c010e2a85 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:40.782686 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:40.854282 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a6411-8cd7-427c-9492-eb7c010e2a85 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:40.889634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992942c-66b7-409d-96ad-5844226cee24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:40.890259 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992942c-66b7-409d-96ad-5844226cee24 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:40.890536 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992942c-66b7-409d-96ad-5844226cee24 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:40.890804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992942c-66b7-409d-96ad-5844226cee24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:40.891531 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992942c-66b7-409d-96ad-5844226cee24 None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=c58bae3ef7784671ae247f4ad4199c0a) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:40.891836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992942c-66b7-409d-96ad-5844226cee24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:40.893253 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992942c-66b7-409d-96ad-5844226cee24 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:40.898595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 625/1260] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:56:40 2022] GET /identity/v3/role_assignments?user.id=c58bae3ef7784671ae247f4ad4199c0a&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:43.056270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429bdc5-55ff-4f2c-8512-a938418025ce None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:43.056813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429bdc5-55ff-4f2c-8512-a938418025ce None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:43.056813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429bdc5-55ff-4f2c-8512-a938418025ce None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:43.058209 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 636/1261] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:56:43.063915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1e784-2bce-4a3f-980a-0ca1706d3ed3 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:43.064321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1e784-2bce-4a3f-980a-0ca1706d3ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:43.064447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1e784-2bce-4a3f-980a-0ca1706d3ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:43.082748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f5c1e784-2bce-4a3f-980a-0ca1706d3ed3 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:43.083918 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f5c1e784-2bce-4a3f-980a-0ca1706d3ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:43.149080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 626/1262] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:56:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:56:43.416107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429bdc5-55ff-4f2c-8512-a938418025ce None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:43.427512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:43.517039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429bdc5-55ff-4f2c-8512-a938418025ce None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:43.551581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928d845-558c-49d7-b294-67e0261b9c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:43.552213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928d845-558c-49d7-b294-67e0261b9c26 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:43.552491 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928d845-558c-49d7-b294-67e0261b9c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:43.552767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928d845-558c-49d7-b294-67e0261b9c26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:43.562668 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928d845-558c-49d7-b294-67e0261b9c26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-4928d845-558c-49d7-b294-67e0261b9c26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:56:43.567487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:43.568615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:43.569702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:43.569702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:43.569702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:43.569702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:43.569702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:56:43.569702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:56:43.569702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:56:43.569702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:56:43.569702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:56:43.569702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:56:43.569702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:56:43.569702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:43.569702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 637/1263] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:56:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:43.576141 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1e784-2bce-4a3f-980a-0ca1706d3ed3 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:43.597981 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:43.699071 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1e784-2bce-4a3f-980a-0ca1706d3ed3 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:43.733024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe04e4-6c36-4a38-827f-c9502ca822c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:43.733619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe04e4-6c36-4a38-827f-c9502ca822c7 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:43.733887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe04e4-6c36-4a38-827f-c9502ca822c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:43.734175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe04e4-6c36-4a38-827f-c9502ca822c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:43.734826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe04e4-6c36-4a38-827f-c9502ca822c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:43.735125 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe04e4-6c36-4a38-827f-c9502ca822c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:43.736479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe04e4-6c36-4a38-827f-c9502ca822c7 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:43.744600 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 627/1264] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:56:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:43.750437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928d845-558c-49d7-b294-67e0261b9c26 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:43.815813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:43.848373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928d845-558c-49d7-b294-67e0261b9c26 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:43.882327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e818fb-771a-4dd4-bdca-ec38012372d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:43.882952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e818fb-771a-4dd4-bdca-ec38012372d4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:43.883226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e818fb-771a-4dd4-bdca-ec38012372d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:43.883493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e818fb-771a-4dd4-bdca-ec38012372d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:43.894042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e818fb-771a-4dd4-bdca-ec38012372d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:43.895497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e818fb-771a-4dd4-bdca-ec38012372d4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-39e818fb-771a-4dd4-bdca-ec38012372d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:56:43.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:43.916020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:56:43.917181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:56:43.917181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:43.917181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:43.917181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 638/1265] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:43.917493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe04e4-6c36-4a38-827f-c9502ca822c7 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:43.997620 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:44.022144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe04e4-6c36-4a38-827f-c9502ca822c7 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:44.055623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7180150-7c48-4e12-b764-2b7d5dc97a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:44.056226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7180150-7c48-4e12-b764-2b7d5dc97a83 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:44.056503 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7180150-7c48-4e12-b764-2b7d5dc97a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:44.056769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7180150-7c48-4e12-b764-2b7d5dc97a83 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:44.057402 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7180150-7c48-4e12-b764-2b7d5dc97a83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:44.057702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7180150-7c48-4e12-b764-2b7d5dc97a83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:44.059082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7180150-7c48-4e12-b764-2b7d5dc97a83 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:44.070637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 628/1266] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:44.076956 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e818fb-771a-4dd4-bdca-ec38012372d4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:44.172909 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e818fb-771a-4dd4-bdca-ec38012372d4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:44.191728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:44.206638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2fd95-897f-4b09-a3a6-eb95e48e2561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:44.207268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2fd95-897f-4b09-a3a6-eb95e48e2561 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:44.207543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2fd95-897f-4b09-a3a6-eb95e48e2561 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:44.207811 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2fd95-897f-4b09-a3a6-eb95e48e2561 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:44.215185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2fd95-897f-4b09-a3a6-eb95e48e2561 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:44.217214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2fd95-897f-4b09-a3a6-eb95e48e2561 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-81a2fd95-897f-4b09-a3a6-eb95e48e2561 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:56:44.228328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:44.229461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:44.230678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:44.230678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:44.230678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:56:44.230678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:56:44.230678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:56:44.230678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:56:44.230678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:56:44.230678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:56:44.230678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:56:44.230678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:44.230678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 639/1267] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:44.235606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7180150-7c48-4e12-b764-2b7d5dc97a83 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:44.331481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7180150-7c48-4e12-b764-2b7d5dc97a83 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:44.365902 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8402b76-5db9-4df9-a8e6-875546134e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:44.366519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8402b76-5db9-4df9-a8e6-875546134e64 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:44.366797 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8402b76-5db9-4df9-a8e6-875546134e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:44.367059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8402b76-5db9-4df9-a8e6-875546134e64 None admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:44.367780 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8402b76-5db9-4df9-a8e6-875546134e64 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:44.368075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8402b76-5db9-4df9-a8e6-875546134e64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:44.369494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8402b76-5db9-4df9-a8e6-875546134e64 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:44.378630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:44.386973 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 629/1268] 10.222.0.62 () {56 vars in 1143 bytes} [Fri Sep 9 11:56:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:44.393859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2fd95-897f-4b09-a3a6-eb95e48e2561 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:44.492139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2fd95-897f-4b09-a3a6-eb95e48e2561 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:44.525624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a085c8-8688-497b-944d-39615cff4831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:44.526279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a085c8-8688-497b-944d-39615cff4831 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:44.526562 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a085c8-8688-497b-944d-39615cff4831 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:44.526835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a085c8-8688-497b-944d-39615cff4831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:44.537129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a085c8-8688-497b-944d-39615cff4831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:44.538599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a085c8-8688-497b-944d-39615cff4831 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-26a085c8-8688-497b-944d-39615cff4831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:56:44.552486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:44.553604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:56:44.554726 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:56:44.554726 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:44.554726 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:44.554726 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 640/1269] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:44.559501 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8402b76-5db9-4df9-a8e6-875546134e64 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:44.662165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8402b76-5db9-4df9-a8e6-875546134e64 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:44.695497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04706c1-1b5a-40a5-9122-e43b851f10d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:44.696088 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04706c1-1b5a-40a5-9122-e43b851f10d1 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:44.696362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04706c1-1b5a-40a5-9122-e43b851f10d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:44.696629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04706c1-1b5a-40a5-9122-e43b851f10d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:44.697275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04706c1-1b5a-40a5-9122-e43b851f10d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:44.697567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04706c1-1b5a-40a5-9122-e43b851f10d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:44.698955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04706c1-1b5a-40a5-9122-e43b851f10d1 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:44.709935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 630/1270] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:44.715707 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a085c8-8688-497b-944d-39615cff4831 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:44.748029 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:44.816130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a085c8-8688-497b-944d-39615cff4831 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:44.848240 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e79e4-2fee-45bd-a639-38bab2c531fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:44.848857 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e79e4-2fee-45bd-a639-38bab2c531fd None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:44.849134 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e79e4-2fee-45bd-a639-38bab2c531fd None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:44.849404 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e79e4-2fee-45bd-a639-38bab2c531fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:44.855838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e79e4-2fee-45bd-a639-38bab2c531fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:44.858166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e79e4-2fee-45bd-a639-38bab2c531fd None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:44.869107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-a74e79e4-2fee-45bd-a639-38bab2c531fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:56:44.869107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:44.869107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:44.869107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:44.869107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:44.869107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:44.869107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:44.869107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:44.869107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:44.869107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:44.869107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:44.869107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:44.869107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:56:44.869107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:56:44.869107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:44.870213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:44.871505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:56:44.871505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:56:44.871505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:56:44.871505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:56:44.871505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:56:44.871505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:56:44.871505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:56:44.871505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:44.871505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 641/1271] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:56:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:44.876744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04706c1-1b5a-40a5-9122-e43b851f10d1 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:44.935929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:44.978932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04706c1-1b5a-40a5-9122-e43b851f10d1 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:45.012721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d131997-13f9-4316-b667-2797dbb3ef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:45.013318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d131997-13f9-4316-b667-2797dbb3ef54 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:45.013597 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d131997-13f9-4316-b667-2797dbb3ef54 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:45.013866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d131997-13f9-4316-b667-2797dbb3ef54 None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:45.014579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d131997-13f9-4316-b667-2797dbb3ef54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:45.014883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d131997-13f9-4316-b667-2797dbb3ef54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:45.016299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d131997-13f9-4316-b667-2797dbb3ef54 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:45.028004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 631/1272] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:56:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:45.036232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e79e4-2fee-45bd-a639-38bab2c531fd None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:45.128538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:45.137394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e79e4-2fee-45bd-a639-38bab2c531fd None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:45.170535 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c17a07-13ae-4168-a73a-28e02d80ff52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:45.171185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c17a07-13ae-4168-a73a-28e02d80ff52 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:45.171456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c17a07-13ae-4168-a73a-28e02d80ff52 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:45.171725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c17a07-13ae-4168-a73a-28e02d80ff52 None admin] PATH_INFO: `/v3/projects/32b06df0c5504592b012585f193386e7/users/c58bae3ef7784671ae247f4ad4199c0a/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:45.173589 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c17a07-13ae-4168-a73a-28e02d80ff52 None admin] RBAC: Authorizing `identity:create_grant(project_id=32b06df0c5504592b012585f193386e7, user_id=c58bae3ef7784671ae247f4ad4199c0a, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:45.175030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c17a07-13ae-4168-a73a-28e02d80ff52 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:45.187280 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 642/1273] 10.222.0.62 () {58 vars in 1439 bytes} [Fri Sep 9 11:56:45 2022] PUT /identity/v3/projects/32b06df0c5504592b012585f193386e7/users/c58bae3ef7784671ae247f4ad4199c0a/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:56:47.328424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3218f71-744e-4aae-ba1d-17b67068aeef None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:47.329240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3218f71-744e-4aae-ba1d-17b67068aeef None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:47.329240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3218f71-744e-4aae-ba1d-17b67068aeef None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:47.330906 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 632/1274] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:56:47.335860 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4e3f1-67c3-47d8-bfe7-6cec793a625c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:47.336241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4e3f1-67c3-47d8-bfe7-6cec793a625c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:47.336419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4e3f1-67c3-47d8-bfe7-6cec793a625c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:47.354725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7fc4e3f1-67c3-47d8-bfe7-6cec793a625c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:47.355905 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7fc4e3f1-67c3-47d8-bfe7-6cec793a625c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:47.420764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 643/1275] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:56:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:56:47.694104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3218f71-744e-4aae-ba1d-17b67068aeef None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:47.793604 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3218f71-744e-4aae-ba1d-17b67068aeef None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:47.811169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:47.828084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b6661-de05-44e3-badc-daf3f72449b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:47.828696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b6661-de05-44e3-badc-daf3f72449b5 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:47.828967 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b6661-de05-44e3-badc-daf3f72449b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:47.829235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b6661-de05-44e3-badc-daf3f72449b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:47.839143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b6661-de05-44e3-badc-daf3f72449b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-ea2b6661-de05-44e3-badc-daf3f72449b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:56:47.844567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:47.845670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:47.846810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:47.846810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:47.846810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:47.846810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:47.846810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:56:47.846810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:56:47.846810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:56:47.846810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:56:47.846810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:56:47.846810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:56:47.846810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:56:47.846810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:47.846810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 633/1276] 10.222.0.62 () {56 vars in 1096 bytes} [Fri Sep 9 11:56:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:47.851813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4e3f1-67c3-47d8-bfe7-6cec793a625c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:47.935369 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:47.954505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4e3f1-67c3-47d8-bfe7-6cec793a625c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:47.987627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39aaa41-a0d2-46ac-aacc-9a06d4159c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:47.988225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39aaa41-a0d2-46ac-aacc-9a06d4159c59 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:47.988500 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39aaa41-a0d2-46ac-aacc-9a06d4159c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:47.988769 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39aaa41-a0d2-46ac-aacc-9a06d4159c59 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:47.989421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39aaa41-a0d2-46ac-aacc-9a06d4159c59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:47.989722 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39aaa41-a0d2-46ac-aacc-9a06d4159c59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:47.991113 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39aaa41-a0d2-46ac-aacc-9a06d4159c59 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:47.999765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 644/1277] 10.222.0.62 () {56 vars in 1099 bytes} [Fri Sep 9 11:56:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:48.005890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b6661-de05-44e3-badc-daf3f72449b5 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:48.103609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b6661-de05-44e3-badc-daf3f72449b5 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:48.136678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1db22-c689-40c8-98ba-4e48bdbb89bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:48.137291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1db22-c689-40c8-98ba-4e48bdbb89bc None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:48.137569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1db22-c689-40c8-98ba-4e48bdbb89bc None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:48.137837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1db22-c689-40c8-98ba-4e48bdbb89bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:48.148201 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1db22-c689-40c8-98ba-4e48bdbb89bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:48.149636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1db22-c689-40c8-98ba-4e48bdbb89bc None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-c9f1db22-c689-40c8-98ba-4e48bdbb89bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:56:48.164075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:48.165098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:56:48.166004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:56:48.166004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:48.166004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:48.166004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 634/1278] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:48.171782 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39aaa41-a0d2-46ac-aacc-9a06d4159c59 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:48.269363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39aaa41-a0d2-46ac-aacc-9a06d4159c59 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:48.302223 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554cb0-660a-4f39-a76a-27c412dbf9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:48.302825 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554cb0-660a-4f39-a76a-27c412dbf9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:48.303100 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554cb0-660a-4f39-a76a-27c412dbf9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:48.303369 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554cb0-660a-4f39-a76a-27c412dbf9c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:48.304004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e554cb0-660a-4f39-a76a-27c412dbf9c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:48.304304 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e554cb0-660a-4f39-a76a-27c412dbf9c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:48.305654 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e554cb0-660a-4f39-a76a-27c412dbf9c7 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:48.316899 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 645/1279] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:48.322623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1db22-c689-40c8-98ba-4e48bdbb89bc None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:48.354559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:48.422894 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1db22-c689-40c8-98ba-4e48bdbb89bc None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:48.454761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb1e42-0a86-49d3-aea4-1a602f82de3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:48.455367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb1e42-0a86-49d3-aea4-1a602f82de3b None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:48.455641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb1e42-0a86-49d3-aea4-1a602f82de3b None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:48.455908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb1e42-0a86-49d3-aea4-1a602f82de3b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:48.463023 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb1e42-0a86-49d3-aea4-1a602f82de3b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:48.464434 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb1e42-0a86-49d3-aea4-1a602f82de3b None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-94cb1e42-0a86-49d3-aea4-1a602f82de3b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:56:48.475843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:48.476974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:48.478361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:48.478361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:48.478361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:56:48.478361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:56:48.478361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:56:48.478361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:56:48.478361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:56:48.478361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:56:48.478361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 11:56:48.478361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:48.478361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 635/1280] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:48.482985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554cb0-660a-4f39-a76a-27c412dbf9c7 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:48.486820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:48.579132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554cb0-660a-4f39-a76a-27c412dbf9c7 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:48.617439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6f3f9-210e-432c-9d18-c5d8e3b1d3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:48.618042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6f3f9-210e-432c-9d18-c5d8e3b1d3cc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:48.618346 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6f3f9-210e-432c-9d18-c5d8e3b1d3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:48.618619 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6f3f9-210e-432c-9d18-c5d8e3b1d3cc None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:48.619341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6f3f9-210e-432c-9d18-c5d8e3b1d3cc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:48.619641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6f3f9-210e-432c-9d18-c5d8e3b1d3cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:48.621061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6f3f9-210e-432c-9d18-c5d8e3b1d3cc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:48.637579 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 646/1281] 10.222.0.62 () {56 vars in 1143 bytes} [Fri Sep 9 11:56:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:48.643906 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb1e42-0a86-49d3-aea4-1a602f82de3b None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:48.734926 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:48.745094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb1e42-0a86-49d3-aea4-1a602f82de3b None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:48.778021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e7aa2-7463-4a0d-a462-305c0dca7915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:48.779237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e7aa2-7463-4a0d-a462-305c0dca7915 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:48.779514 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e7aa2-7463-4a0d-a462-305c0dca7915 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:48.779785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e7aa2-7463-4a0d-a462-305c0dca7915 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:48.845405 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e7aa2-7463-4a0d-a462-305c0dca7915 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:48.846863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e7aa2-7463-4a0d-a462-305c0dca7915 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:48.932675 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:48.943798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 636/1282] 10.222.0.62 () {58 vars in 1282 bytes} [Fri Sep 9 11:56:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 11:56:48.949607 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6f3f9-210e-432c-9d18-c5d8e3b1d3cc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:49.009321 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:49.047231 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6f3f9-210e-432c-9d18-c5d8e3b1d3cc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:49.081462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a914c-24dd-47a7-b266-087e9480bc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:49.082083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a914c-24dd-47a7-b266-087e9480bc1a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:49.082389 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a914c-24dd-47a7-b266-087e9480bc1a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:49.082655 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a914c-24dd-47a7-b266-087e9480bc1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:49.092428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a914c-24dd-47a7-b266-087e9480bc1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:49.093852 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a914c-24dd-47a7-b266-087e9480bc1a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-1c4a914c-24dd-47a7-b266-087e9480bc1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:56:49.108531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:49.109691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:56:49.110677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:56:49.110677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:56:49.110677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:49.110677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 647/1283] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 11:56:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:49.115654 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e7aa2-7463-4a0d-a462-305c0dca7915 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:49.214074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e7aa2-7463-4a0d-a462-305c0dca7915 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:49.248944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031eedd4-4ca1-46b3-a95e-dc686a7ea635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:49.249582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031eedd4-4ca1-46b3-a95e-dc686a7ea635 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:49.249858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031eedd4-4ca1-46b3-a95e-dc686a7ea635 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:49.250157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031eedd4-4ca1-46b3-a95e-dc686a7ea635 None admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:49.250801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031eedd4-4ca1-46b3-a95e-dc686a7ea635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:49.251103 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031eedd4-4ca1-46b3-a95e-dc686a7ea635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:49.252461 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031eedd4-4ca1-46b3-a95e-dc686a7ea635 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:49.263825 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 637/1284] 10.222.0.62 () {56 vars in 1109 bytes} [Fri Sep 9 11:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:49.269540 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a914c-24dd-47a7-b266-087e9480bc1a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:49.365907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a914c-24dd-47a7-b266-087e9480bc1a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:49.382999 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:49.404923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8956d2-1671-422a-bef2-816299aade8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:49.405554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8956d2-1671-422a-bef2-816299aade8d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:49.405835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8956d2-1671-422a-bef2-816299aade8d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:49.406107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8956d2-1671-422a-bef2-816299aade8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:49.412717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8956d2-1671-422a-bef2-816299aade8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:49.414171 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8956d2-1671-422a-bef2-816299aade8d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:49.425861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-8b8956d2-1671-422a-bef2-816299aade8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:56:49.425861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:49.425861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:49.425861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:49.425861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:49.425861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:49.425861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:49.425861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:49.425861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:49.425861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:49.425861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:49.425861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:49.425861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:56:49.425861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:56:49.425861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:49.427169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:49.428363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:56:49.428363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:56:49.428363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:56:49.428363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:56:49.428363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:56:49.428363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:56:49.428363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:56:49.428363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 09 11:56:49.428363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 648/1285] 10.222.0.62 () {56 vars in 1111 bytes} [Fri Sep 9 11:56:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:49.432891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031eedd4-4ca1-46b3-a95e-dc686a7ea635 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:49.482449 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:49.532176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031eedd4-4ca1-46b3-a95e-dc686a7ea635 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:49.565393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b277c9-949e-4461-b05f-08e9e6d5194c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:49.565992 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b277c9-949e-4461-b05f-08e9e6d5194c None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:49.566301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b277c9-949e-4461-b05f-08e9e6d5194c None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:49.566575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b277c9-949e-4461-b05f-08e9e6d5194c None admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:49.567261 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b277c9-949e-4461-b05f-08e9e6d5194c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:49.567567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b277c9-949e-4461-b05f-08e9e6d5194c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:49.568987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b277c9-949e-4461-b05f-08e9e6d5194c None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:49.580831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 638/1286] 10.222.0.62 () {56 vars in 1148 bytes} [Fri Sep 9 11:56:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:49.586897 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8956d2-1671-422a-bef2-816299aade8d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:49.684040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8956d2-1671-422a-bef2-816299aade8d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:49.716535 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde6a60-9188-46d6-998e-2ecfa39328b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:49.717130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde6a60-9188-46d6-998e-2ecfa39328b0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:49.717407 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde6a60-9188-46d6-998e-2ecfa39328b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:49.717679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde6a60-9188-46d6-998e-2ecfa39328b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:49.718429 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde6a60-9188-46d6-998e-2ecfa39328b0 None admin] RBAC: Adding query filter params (role.id=a57eb61267c6490c8cb77fb1e31926dc, scope.project.id=32b06df0c5504592b012585f193386e7, user.id=c58bae3ef7784671ae247f4ad4199c0a) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:49.718736 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde6a60-9188-46d6-998e-2ecfa39328b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:49.720165 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde6a60-9188-46d6-998e-2ecfa39328b0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:49.726381 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 649/1287] 10.222.0.62 () {56 vars in 1374 bytes} [Fri Sep 9 11:56:49 2022] GET /identity/v3/role_assignments?user.id=c58bae3ef7784671ae247f4ad4199c0a&scope.project.id=32b06df0c5504592b012585f193386e7&role.id=a57eb61267c6490c8cb77fb1e31926dc => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:51.874451 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1a2e2-c35d-45f5-9982-620dbd3fd2d9 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:51.875000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1a2e2-c35d-45f5-9982-620dbd3fd2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:51.875000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1a2e2-c35d-45f5-9982-620dbd3fd2d9 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:51.876314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 639/1288] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:56:51.881668 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d09242-d81a-47eb-8b92-66c013eba291 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:51.882150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d09242-d81a-47eb-8b92-66c013eba291 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:51.882245 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d09242-d81a-47eb-8b92-66c013eba291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:51.900108 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f6d09242-d81a-47eb-8b92-66c013eba291 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:51.901302 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f6d09242-d81a-47eb-8b92-66c013eba291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:51.930485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:51.966945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 650/1289] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:56:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:56:52.239780 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1a2e2-c35d-45f5-9982-620dbd3fd2d9 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:52.339026 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1a2e2-c35d-45f5-9982-620dbd3fd2d9 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:52.350189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:52.373544 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75e977-5460-41dc-bfbe-a5c24c478b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:52.374187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75e977-5460-41dc-bfbe-a5c24c478b50 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:52.374468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75e977-5460-41dc-bfbe-a5c24c478b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:52.374735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75e977-5460-41dc-bfbe-a5c24c478b50 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:52.375318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75e977-5460-41dc-bfbe-a5c24c478b50 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:52.376677 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75e977-5460-41dc-bfbe-a5c24c478b50 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-ae75e977-5460-41dc-bfbe-a5c24c478b50 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:52.385346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:52.386569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:52.387457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:52.387457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:52.387457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:52.387457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:52.387457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:52.387457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:52.387457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:52.387457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:52.387457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:52.387457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:52.387457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:56:52.387457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:56:52.387457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:56:52.387457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:52.387457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 640/1290] 10.222.0.62 () {56 vars in 1117 bytes} [Fri Sep 9 11:56:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:52.393189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d09242-d81a-47eb-8b92-66c013eba291 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:52.487444 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d09242-d81a-47eb-8b92-66c013eba291 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:52.519865 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d535d6-8f59-44d3-8d34-0645b974b8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:52.520454 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d535d6-8f59-44d3-8d34-0645b974b8bd None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:52.520726 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d535d6-8f59-44d3-8d34-0645b974b8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:52.520993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d535d6-8f59-44d3-8d34-0645b974b8bd None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:52.521607 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d535d6-8f59-44d3-8d34-0645b974b8bd None admin] RBAC: Adding query filter params (name=placement) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:52.521903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d535d6-8f59-44d3-8d34-0645b974b8bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:52.523278 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d535d6-8f59-44d3-8d34-0645b974b8bd None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:52.527882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 651/1291] 10.222.0.62 () {56 vars in 1116 bytes} [Fri Sep 9 11:56:52 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:52.534247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75e977-5460-41dc-bfbe-a5c24c478b50 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:52.649965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75e977-5460-41dc-bfbe-a5c24c478b50 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:52.683187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b346ce-7843-4987-8f1c-8e5cc02c7d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:52.683785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b346ce-7843-4987-8f1c-8e5cc02c7d43 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:52.684061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b346ce-7843-4987-8f1c-8e5cc02c7d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:52.684327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b346ce-7843-4987-8f1c-8e5cc02c7d43 None admin] PATH_INFO: `/v3/services` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:52.685459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b346ce-7843-4987-8f1c-8e5cc02c7d43 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:52.685759 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b346ce-7843-4987-8f1c-8e5cc02c7d43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:52.687155 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b346ce-7843-4987-8f1c-8e5cc02c7d43 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:52.691423 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 641/1292] 10.222.0.62 () {56 vars in 1116 bytes} [Fri Sep 9 11:56:52 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:54.818744 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6de34-dfe6-40f1-bcb5-c0f755c18c1d None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:54.819307 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6de34-dfe6-40f1-bcb5-c0f755c18c1d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:54.819307 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6de34-dfe6-40f1-bcb5-c0f755c18c1d None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:54.820627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 652/1293] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:56:54.826395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c17834-f27d-462d-8a3b-062e8c358487 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:54.826822 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c17834-f27d-462d-8a3b-062e8c358487 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:54.826924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c17834-f27d-462d-8a3b-062e8c358487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:54.844902 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-b0c17834-f27d-462d-8a3b-062e8c358487 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:54.846138 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b0c17834-f27d-462d-8a3b-062e8c358487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:54.875853 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:54.913875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 642/1294] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:56:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:56:55.188396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6de34-dfe6-40f1-bcb5-c0f755c18c1d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:55.218559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:55.290087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6de34-dfe6-40f1-bcb5-c0f755c18c1d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:55.323685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d91032-5fd4-42de-8ee2-11e9efd8c021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:55.324276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d91032-5fd4-42de-8ee2-11e9efd8c021 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:55.324548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d91032-5fd4-42de-8ee2-11e9efd8c021 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:55.324813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d91032-5fd4-42de-8ee2-11e9efd8c021 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:55.325458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d91032-5fd4-42de-8ee2-11e9efd8c021 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:55.326824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d91032-5fd4-42de-8ee2-11e9efd8c021 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:55.341894 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 653/1295] 10.222.0.62 () {60 vars in 1141 bytes} [Fri Sep 9 11:56:55 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:56:57.510280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27850b2b-05f5-45d7-8d92-a5af747f6ae0 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:57.510793 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27850b2b-05f5-45d7-8d92-a5af747f6ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:57.510793 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27850b2b-05f5-45d7-8d92-a5af747f6ae0 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:57.512089 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 643/1296] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:56:57.518061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775b3b0-717c-4921-9adf-b7b1679b89d4 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:57.518530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775b3b0-717c-4921-9adf-b7b1679b89d4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:57.518640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775b3b0-717c-4921-9adf-b7b1679b89d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:57.537004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-0775b3b0-717c-4921-9adf-b7b1679b89d4 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:56:57.538233 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0775b3b0-717c-4921-9adf-b7b1679b89d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:56:57.616010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 654/1297] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:56:57 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 11:56:57.895757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27850b2b-05f5-45d7-8d92-a5af747f6ae0 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:57.994481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27850b2b-05f5-45d7-8d92-a5af747f6ae0 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:58.029124 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e46b2c-9779-42a8-bb21-e3c6138ca705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:58.029784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e46b2c-9779-42a8-bb21-e3c6138ca705 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:58.030058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e46b2c-9779-42a8-bb21-e3c6138ca705 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:58.030357 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e46b2c-9779-42a8-bb21-e3c6138ca705 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:58.030956 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e46b2c-9779-42a8-bb21-e3c6138ca705 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:58.032337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e46b2c-9779-42a8-bb21-e3c6138ca705 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-96e46b2c-9779-42a8-bb21-e3c6138ca705 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:56:58.047444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:56:58.049343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:56:58.051365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:56:58.051365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:56:58.051365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:56:58.051365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:56:58.051365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:56:58.051365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:56:58.051365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:56:58.051365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:56:58.051365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:56:58.051365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:56:58.051365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:56:58.051365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:56:58.051365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:56:58.051365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:56:58.051365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 644/1298] 10.222.0.62 () {56 vars in 1117 bytes} [Fri Sep 9 11:56:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:56:58.056463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775b3b0-717c-4921-9adf-b7b1679b89d4 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:58.075388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:58.157265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775b3b0-717c-4921-9adf-b7b1679b89d4 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:58.190262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422065d-3eaf-450c-905e-74530141ff5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:58.190858 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422065d-3eaf-450c-905e-74530141ff5c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:58.191131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422065d-3eaf-450c-905e-74530141ff5c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:58.191395 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422065d-3eaf-450c-905e-74530141ff5c None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:58.192020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422065d-3eaf-450c-905e-74530141ff5c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:58.192315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422065d-3eaf-450c-905e-74530141ff5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:58.193665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422065d-3eaf-450c-905e-74530141ff5c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:58.199052 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 655/1299] 10.222.0.62 () {56 vars in 1116 bytes} [Fri Sep 9 11:56:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:58.204943 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e46b2c-9779-42a8-bb21-e3c6138ca705 None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:58.236382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:58.302351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e46b2c-9779-42a8-bb21-e3c6138ca705 None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:58.334457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f3581-af0d-494c-b84b-42eafc70c9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:58.335051 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f3581-af0d-494c-b84b-42eafc70c9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:58.335318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f3581-af0d-494c-b84b-42eafc70c9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:58.335582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f3581-af0d-494c-b84b-42eafc70c9e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:58.336267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f3581-af0d-494c-b84b-42eafc70c9e2 None admin] RBAC: Adding query filter params (interface=public, service_id=d573091b827645a4bee771298d968b03, region_id=RegionOne) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:56:58.336566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f3581-af0d-494c-b84b-42eafc70c9e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:58.337911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f3581-af0d-494c-b84b-42eafc70c9e2 None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:58.343686 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 645/1300] 10.222.0.62 () {56 vars in 1251 bytes} [Fri Sep 9 11:56:58 2022] GET /identity/v3/endpoints?service_id=d573091b827645a4bee771298d968b03&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:56:58.349393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422065d-3eaf-450c-905e-74530141ff5c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:56:58.435325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:56:58.453239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422065d-3eaf-450c-905e-74530141ff5c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:56:58.485810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eae248-bb99-47b8-9097-a171bae287d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:56:58.486442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eae248-bb99-47b8-9097-a171bae287d3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:56:58.486724 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eae248-bb99-47b8-9097-a171bae287d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:56:58.486995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eae248-bb99-47b8-9097-a171bae287d3 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:56:58.487595 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eae248-bb99-47b8-9097-a171bae287d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:56:58.488942 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eae248-bb99-47b8-9097-a171bae287d3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:56:58.495588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 656/1301] 10.222.0.62 () {56 vars in 1087 bytes} [Fri Sep 9 11:56:58 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 11:57:00.668997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfeee90-51fa-4ee2-a460-35e8d4b9b504 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:00.669589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfeee90-51fa-4ee2-a460-35e8d4b9b504 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:00.669589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfeee90-51fa-4ee2-a460-35e8d4b9b504 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:00.670905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 646/1302] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:57:00.675980 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98a604-b396-4c6e-80ca-cdfe5721feae None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:00.676432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98a604-b396-4c6e-80ca-cdfe5721feae None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:00.676534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98a604-b396-4c6e-80ca-cdfe5721feae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:00.694613 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-8b98a604-b396-4c6e-80ca-cdfe5721feae None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:57:00.695793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8b98a604-b396-4c6e-80ca-cdfe5721feae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:57:00.760611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 657/1303] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:57:00 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 11:57:01.028107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfeee90-51fa-4ee2-a460-35e8d4b9b504 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:01.120274 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:01.127539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfeee90-51fa-4ee2-a460-35e8d4b9b504 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:01.160827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af965e16-9836-4034-985f-5a47ecaccf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:01.161453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af965e16-9836-4034-985f-5a47ecaccf5f None admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:01.161737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af965e16-9836-4034-985f-5a47ecaccf5f None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:01.162009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af965e16-9836-4034-985f-5a47ecaccf5f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:01.162626 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af965e16-9836-4034-985f-5a47ecaccf5f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:01.163985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af965e16-9836-4034-985f-5a47ecaccf5f None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application [None req-af965e16-9836-4034-985f-5a47ecaccf5f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:57:01.173139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:57:01.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:57:01.175358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:57:01.175358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application with Lock( Sep 09 11:57:01.175358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:57:01.175358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:57:01.175358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:57:01.175358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:57:01.175358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:57:01.175358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:57:01.175358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:57:01.175358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:57:01.175358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 11:57:01.175358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 11:57:01.175358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 11:57:01.175358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: ERROR keystone.server.flask.application Sep 09 11:57:01.175358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 647/1304] 10.222.0.62 () {56 vars in 1117 bytes} [Fri Sep 9 11:57:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:57:01.180404 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98a604-b396-4c6e-80ca-cdfe5721feae None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:01.255437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:01.277478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98a604-b396-4c6e-80ca-cdfe5721feae None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:01.310579 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b34cca-8616-4413-94c2-d89c8ba419fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:01.311114 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b34cca-8616-4413-94c2-d89c8ba419fa None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:01.311388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b34cca-8616-4413-94c2-d89c8ba419fa None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:01.311657 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b34cca-8616-4413-94c2-d89c8ba419fa None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:01.312274 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b34cca-8616-4413-94c2-d89c8ba419fa None admin] RBAC: Adding query filter params (name=placement) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:57:01.312581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b34cca-8616-4413-94c2-d89c8ba419fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:01.313945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b34cca-8616-4413-94c2-d89c8ba419fa None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:01.319014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 658/1305] 10.222.0.62 () {56 vars in 1116 bytes} [Fri Sep 9 11:57:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:57:01.324978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af965e16-9836-4034-985f-5a47ecaccf5f None admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:57:01.420921 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af965e16-9836-4034-985f-5a47ecaccf5f None admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:57:01.455570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e873b-dfd4-4ec9-9216-7470972ae03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:57:01.456172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e873b-dfd4-4ec9-9216-7470972ae03d None admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:57:01.456446 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e873b-dfd4-4ec9-9216-7470972ae03d None admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:57:01.456708 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e873b-dfd4-4ec9-9216-7470972ae03d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:57:01.457375 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e873b-dfd4-4ec9-9216-7470972ae03d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:57:01.458761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e873b-dfd4-4ec9-9216-7470972ae03d None admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:57:01.478665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 648/1306] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 11:57:01 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:57:11.315966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=30 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:57:11.475529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:06.233153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999a996-cb1a-40c0-b4df-8a27e47d4961 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:06.233611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999a996-cb1a-40c0-b4df-8a27e47d4961 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:06.233611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999a996-cb1a-40c0-b4df-8a27e47d4961 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:06.238982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 659/1307] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:58:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:06.240249 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf4163-fe23-4737-9171-6fe23fbb86b1 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:06.240668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf4163-fe23-4737-9171-6fe23fbb86b1 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:06.240902 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf4163-fe23-4737-9171-6fe23fbb86b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:06.411862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d2bf4163-fe23-4737-9171-6fe23fbb86b1 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:06.415111 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d2bf4163-fe23-4737-9171-6fe23fbb86b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:06.558164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 649/1308] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:06.862449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a0ebe-215f-4854-820c-df264352d8d6 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:06.862743 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a0ebe-215f-4854-820c-df264352d8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:06.863012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a0ebe-215f-4854-820c-df264352d8d6 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:06.864721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 660/1309] 10.222.0.62 () {54 vars in 965 bytes} [Fri Sep 9 11:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:06.871376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3617ba-824f-4aa7-ad83-67da74d9be71 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:06.871666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3617ba-824f-4aa7-ad83-67da74d9be71 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:06.871943 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3617ba-824f-4aa7-ad83-67da74d9be71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:06.958234 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-bc3617ba-824f-4aa7-ad83-67da74d9be71 None None] MFA Rules not processed for user `d00634cdd3fb41e794d3af0b88ba7338`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:06.959514 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-bc3617ba-824f-4aa7-ad83-67da74d9be71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:06.987150 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:07.054157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 650/1310] 10.222.0.62 () {58 vars in 1062 bytes} [Fri Sep 9 11:58:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:07.064142 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a0ebe-215f-4854-820c-df264352d8d6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:07.065922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c23a0ebe-215f-4854-820c-df264352d8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:07.200483 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a0ebe-215f-4854-820c-df264352d8d6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:07.288050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f32922-ebf2-4f86-9a1e-76407ff1f394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:07.288771 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f32922-ebf2-4f86-9a1e-76407ff1f394 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:07.289060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f32922-ebf2-4f86-9a1e-76407ff1f394 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:07.289233 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f32922-ebf2-4f86-9a1e-76407ff1f394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:07.291393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c5f32922-ebf2-4f86-9a1e-76407ff1f394 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:07.350840 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:07.411285 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f32922-ebf2-4f86-9a1e-76407ff1f394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:07.412601 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f32922-ebf2-4f86-9a1e-76407ff1f394 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:07.433176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 661/1311] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:09.899894 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48005d2-a589-46cd-bd6c-b87925e173d1 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:09.900431 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48005d2-a589-46cd-bd6c-b87925e173d1 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:09.900543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48005d2-a589-46cd-bd6c-b87925e173d1 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:09.901802 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 651/1312] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:09.907682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28f45e-45a2-4064-836f-370f56fceb3a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:09.908105 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28f45e-45a2-4064-836f-370f56fceb3a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:09.908419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28f45e-45a2-4064-836f-370f56fceb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:09.993558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ed28f45e-45a2-4064-836f-370f56fceb3a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:09.994788 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ed28f45e-45a2-4064-836f-370f56fceb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:10.090243 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 662/1313] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:58:10.378019 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48005d2-a589-46cd-bd6c-b87925e173d1 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:10.379596 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b48005d2-a589-46cd-bd6c-b87925e173d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:10.487000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48005d2-a589-46cd-bd6c-b87925e173d1 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:10.564089 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:10.567007 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18394fd2-f646-49c2-b8e5-aa4cb3218a9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:10.567906 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18394fd2-f646-49c2-b8e5-aa4cb3218a9e service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:10.568421 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18394fd2-f646-49c2-b8e5-aa4cb3218a9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:10.568702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18394fd2-f646-49c2-b8e5-aa4cb3218a9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:10.570596 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-18394fd2-f646-49c2-b8e5-aa4cb3218a9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:10.683230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18394fd2-f646-49c2-b8e5-aa4cb3218a9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:10.684693 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18394fd2-f646-49c2-b8e5-aa4cb3218a9e service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:10.703307 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 652/1314] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:13.183648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed6067-6df1-47a8-892e-70f329339aff None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:13.184187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed6067-6df1-47a8-892e-70f329339aff None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:13.184316 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed6067-6df1-47a8-892e-70f329339aff None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:13.185607 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 663/1315] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:13.190913 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc35705-1b0b-41d3-9570-4a5a1a8ac029 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:13.191364 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc35705-1b0b-41d3-9570-4a5a1a8ac029 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:13.191462 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc35705-1b0b-41d3-9570-4a5a1a8ac029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:13.259342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-efc35705-1b0b-41d3-9570-4a5a1a8ac029 None None] MFA Rules not processed for user `b7b49d2a1ef74401b3242303d28a3f46`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:13.260557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-efc35705-1b0b-41d3-9570-4a5a1a8ac029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:13.330358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:13.375702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 653/1316] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:13 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:58:13.684980 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d73757-3f5e-47f2-bee6-0819c2899046 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:13.685262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d73757-3f5e-47f2-bee6-0819c2899046 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:13.685535 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d73757-3f5e-47f2-bee6-0819c2899046 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:13.686915 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 664/1317] 10.222.0.62 () {54 vars in 965 bytes} [Fri Sep 9 11:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:13.693187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c0055-24ed-4709-9a95-f7c5b51498ea None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:13.693467 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c0055-24ed-4709-9a95-f7c5b51498ea None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:13.693734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c0055-24ed-4709-9a95-f7c5b51498ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:13.773747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-4d7c0055-24ed-4709-9a95-f7c5b51498ea None None] MFA Rules not processed for user `d00634cdd3fb41e794d3af0b88ba7338`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:13.774994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-4d7c0055-24ed-4709-9a95-f7c5b51498ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:13.850334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 654/1318] 10.222.0.62 () {58 vars in 1062 bytes} [Fri Sep 9 11:58:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:13.860191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d73757-3f5e-47f2-bee6-0819c2899046 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:13.861734 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-77d73757-3f5e-47f2-bee6-0819c2899046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:13.872200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:13.965942 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d73757-3f5e-47f2-bee6-0819c2899046 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:14.037827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ddb954-b951-48d3-bf35-fad274296514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:14.038472 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddb954-b951-48d3-bf35-fad274296514 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:14.038752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddb954-b951-48d3-bf35-fad274296514 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:14.039025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddb954-b951-48d3-bf35-fad274296514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:14.041194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-77ddb954-b951-48d3-bf35-fad274296514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:14.106746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:14.156672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ddb954-b951-48d3-bf35-fad274296514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:14.158137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ddb954-b951-48d3-bf35-fad274296514 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:14.177372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 665/1319] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:13 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:17.259281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbb97d-d142-44ee-b514-c16b070d010a None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:17.259875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbb97d-d142-44ee-b514-c16b070d010a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:17.259875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbb97d-d142-44ee-b514-c16b070d010a None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:17.261185 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 655/1320] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:17.266482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876e2e5-a9e6-43c8-b879-4ec69c3f467d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:17.266912 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876e2e5-a9e6-43c8-b879-4ec69c3f467d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:17.267038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876e2e5-a9e6-43c8-b879-4ec69c3f467d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:17.333380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-6876e2e5-a9e6-43c8-b879-4ec69c3f467d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:17.334614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6876e2e5-a9e6-43c8-b879-4ec69c3f467d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:17.430464 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 666/1321] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:17.711495 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbb97d-d142-44ee-b514-c16b070d010a None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:17.713164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b7fbb97d-d142-44ee-b514-c16b070d010a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:17.764916 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:17.820791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbb97d-d142-44ee-b514-c16b070d010a None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:17.896664 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453f871-6a12-4b25-a03c-96525ccf1590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:17.897264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453f871-6a12-4b25-a03c-96525ccf1590 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:17.897539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453f871-6a12-4b25-a03c-96525ccf1590 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:17.897808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453f871-6a12-4b25-a03c-96525ccf1590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:17.900159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e453f871-6a12-4b25-a03c-96525ccf1590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:17.995858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:18.009490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453f871-6a12-4b25-a03c-96525ccf1590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:18.010955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453f871-6a12-4b25-a03c-96525ccf1590 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:18.029046 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 656/1322] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:20.861337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61696149-8468-446f-99c0-88cd1e6f8c9b None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:20.861882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61696149-8468-446f-99c0-88cd1e6f8c9b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:20.861882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61696149-8468-446f-99c0-88cd1e6f8c9b None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:20.863277 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 667/1323] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:20.868548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0d8c3-eb37-4b26-91e0-66556172f512 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:20.869014 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0d8c3-eb37-4b26-91e0-66556172f512 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:20.869106 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0d8c3-eb37-4b26-91e0-66556172f512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:20.936081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-05d0d8c3-eb37-4b26-91e0-66556172f512 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:20.937292 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-05d0d8c3-eb37-4b26-91e0-66556172f512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:21.050944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 657/1324] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:21.339892 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61696149-8468-446f-99c0-88cd1e6f8c9b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:21.341452 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-61696149-8468-446f-99c0-88cd1e6f8c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:21.363221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:21.451121 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61696149-8468-446f-99c0-88cd1e6f8c9b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:21.526013 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81ff89-7c8a-4e83-aacd-d840257e01a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:21.526686 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81ff89-7c8a-4e83-aacd-d840257e01a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:21.526963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81ff89-7c8a-4e83-aacd-d840257e01a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:21.527230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81ff89-7c8a-4e83-aacd-d840257e01a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:21.528867 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8d81ff89-7c8a-4e83-aacd-d840257e01a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:21.604769 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:21.643675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81ff89-7c8a-4e83-aacd-d840257e01a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:21.645024 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81ff89-7c8a-4e83-aacd-d840257e01a5 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:21.664870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 668/1325] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:23.890152 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252e96-63a1-44e2-ae0c-ff4828e7b1e3 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:23.890764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252e96-63a1-44e2-ae0c-ff4828e7b1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:23.890764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2252e96-63a1-44e2-ae0c-ff4828e7b1e3 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:23.892242 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 658/1326] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:58:23.897414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ee9ba-e21f-460c-a699-84b4753bdefb None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:23.897829 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ee9ba-e21f-460c-a699-84b4753bdefb None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:23.897978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ee9ba-e21f-460c-a699-84b4753bdefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:23.966980 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b58ee9ba-e21f-460c-a699-84b4753bdefb None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:23.968165 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b58ee9ba-e21f-460c-a699-84b4753bdefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:24.072295 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 669/1327] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:24.356083 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2252e96-63a1-44e2-ae0c-ff4828e7b1e3 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:24.357634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b2252e96-63a1-44e2-ae0c-ff4828e7b1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:24.423457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:24.468110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2252e96-63a1-44e2-ae0c-ff4828e7b1e3 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:24.548326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6523f6-9d5f-4057-a6bb-d04a8cee5591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:24.548939 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6523f6-9d5f-4057-a6bb-d04a8cee5591 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:24.549216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6523f6-9d5f-4057-a6bb-d04a8cee5591 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:24.549486 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6523f6-9d5f-4057-a6bb-d04a8cee5591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:24.551186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-9f6523f6-9d5f-4057-a6bb-d04a8cee5591 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:24.667398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:24.670060 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6523f6-9d5f-4057-a6bb-d04a8cee5591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:24.672115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6523f6-9d5f-4057-a6bb-d04a8cee5591 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:24.691531 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 659/1328] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:26.934470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a41604-9cd2-4826-9e5a-622189e898b4 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:26.934985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a41604-9cd2-4826-9e5a-622189e898b4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:26.934985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a41604-9cd2-4826-9e5a-622189e898b4 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:26.936320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 670/1329] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:26.942086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d5609-6726-4972-acf9-a3918514dcb6 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:26.942629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d5609-6726-4972-acf9-a3918514dcb6 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:26.942725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d5609-6726-4972-acf9-a3918514dcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:27.016856 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-0b7d5609-6726-4972-acf9-a3918514dcb6 None None] MFA Rules not processed for user `b7b49d2a1ef74401b3242303d28a3f46`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:27.018025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0b7d5609-6726-4972-acf9-a3918514dcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:27.120148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 660/1330] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:26 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:27.399157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a41604-9cd2-4826-9e5a-622189e898b4 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:27.400724 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-57a41604-9cd2-4826-9e5a-622189e898b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:27.435435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:27.514693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a41604-9cd2-4826-9e5a-622189e898b4 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:27.593364 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bab1bb-1288-4692-9f7d-09b9010662cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:27.593982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bab1bb-1288-4692-9f7d-09b9010662cb service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:27.594288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bab1bb-1288-4692-9f7d-09b9010662cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:27.594556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bab1bb-1288-4692-9f7d-09b9010662cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:27.596155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d9bab1bb-1288-4692-9f7d-09b9010662cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:27.678925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:27.712292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bab1bb-1288-4692-9f7d-09b9010662cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:27.713743 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bab1bb-1288-4692-9f7d-09b9010662cb service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:27.733471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 671/1331] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:27 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:58:30.195497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424b999-e469-487c-8133-36335a63981d None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:30.196038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424b999-e469-487c-8133-36335a63981d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:30.196038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424b999-e469-487c-8133-36335a63981d None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:30.197398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 661/1332] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:30.203090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9a127-781b-4ead-aa43-0a4a351bd8c2 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:30.203843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9a127-781b-4ead-aa43-0a4a351bd8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:30.204026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9a127-781b-4ead-aa43-0a4a351bd8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:30.273589 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9dd9a127-781b-4ead-aa43-0a4a351bd8c2 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:30.274807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9dd9a127-781b-4ead-aa43-0a4a351bd8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:30.380808 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 672/1333] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:30.669352 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424b999-e469-487c-8133-36335a63981d None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:30.670921 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-9424b999-e469-487c-8133-36335a63981d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:30.745531 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:30.780610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424b999-e469-487c-8133-36335a63981d None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:30.854050 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8cabf-4ecc-4e32-a506-2ec53f9ff240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:30.854965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8cabf-4ecc-4e32-a506-2ec53f9ff240 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:30.855235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8cabf-4ecc-4e32-a506-2ec53f9ff240 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:30.855500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8cabf-4ecc-4e32-a506-2ec53f9ff240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:30.857090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-73b8cabf-4ecc-4e32-a506-2ec53f9ff240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:30.966907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8cabf-4ecc-4e32-a506-2ec53f9ff240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:30.968246 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8cabf-4ecc-4e32-a506-2ec53f9ff240 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:30.978545 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:30.993960 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 662/1334] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:33.270969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95fbaa-270a-4d2f-93b5-4cafda4d96e0 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:33.271581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95fbaa-270a-4d2f-93b5-4cafda4d96e0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:33.271581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95fbaa-270a-4d2f-93b5-4cafda4d96e0 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:33.273376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 673/1335] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:58:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:33.278818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8946412-7498-42c5-80a4-3b4aed56d833 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:33.279287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8946412-7498-42c5-80a4-3b4aed56d833 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:33.279393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8946412-7498-42c5-80a4-3b4aed56d833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:33.345808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d8946412-7498-42c5-80a4-3b4aed56d833 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:33.347061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d8946412-7498-42c5-80a4-3b4aed56d833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:33.443509 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 663/1336] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:58:33.733598 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95fbaa-270a-4d2f-93b5-4cafda4d96e0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:33.735453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0e95fbaa-270a-4d2f-93b5-4cafda4d96e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:33.783156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:33.847322 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95fbaa-270a-4d2f-93b5-4cafda4d96e0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:33.921568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28d9be-99c8-4a42-aab0-51830976b13c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:33.922201 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28d9be-99c8-4a42-aab0-51830976b13c service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:33.922478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28d9be-99c8-4a42-aab0-51830976b13c service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:33.922746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28d9be-99c8-4a42-aab0-51830976b13c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:33.924332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ba28d9be-99c8-4a42-aab0-51830976b13c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:34.006972 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:34.041935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28d9be-99c8-4a42-aab0-51830976b13c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:34.043398 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28d9be-99c8-4a42-aab0-51830976b13c service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:34.063004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 674/1337] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:36.693510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f52743-b5c9-4850-a4d1-f3e40104751d None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:36.694627 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f52743-b5c9-4850-a4d1-f3e40104751d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:36.694627 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f52743-b5c9-4850-a4d1-f3e40104751d None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:36.695966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 664/1338] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:36.704990 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4dea5-be23-454a-924d-b1d87ad40e5e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:36.705545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4dea5-be23-454a-924d-b1d87ad40e5e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:36.705913 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4dea5-be23-454a-924d-b1d87ad40e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:36.785431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-aee4dea5-be23-454a-924d-b1d87ad40e5e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:36.786664 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-aee4dea5-be23-454a-924d-b1d87ad40e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:36.885042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 675/1339] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:37.171559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f52743-b5c9-4850-a4d1-f3e40104751d None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:37.173119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a8f52743-b5c9-4850-a4d1-f3e40104751d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:37.251528 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:37.279507 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f52743-b5c9-4850-a4d1-f3e40104751d None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:37.353436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998f4d3-1476-4d0c-9869-cedc6c809be6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:37.354036 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998f4d3-1476-4d0c-9869-cedc6c809be6 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:37.354609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998f4d3-1476-4d0c-9869-cedc6c809be6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:37.354879 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998f4d3-1476-4d0c-9869-cedc6c809be6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:37.356488 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-3998f4d3-1476-4d0c-9869-cedc6c809be6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:37.467271 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998f4d3-1476-4d0c-9869-cedc6c809be6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:37.468617 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998f4d3-1476-4d0c-9869-cedc6c809be6 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:37.487045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 665/1340] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:41.628246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f1736-a4a9-4fc4-a99b-9d0edb217221 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:41.628809 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f1736-a4a9-4fc4-a99b-9d0edb217221 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:41.628809 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f1736-a4a9-4fc4-a99b-9d0edb217221 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:41.630186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 676/1341] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:41.635536 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db1969-ad78-4390-aa1d-0f75e6824d5a None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:41.636008 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db1969-ad78-4390-aa1d-0f75e6824d5a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:41.636123 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db1969-ad78-4390-aa1d-0f75e6824d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:41.641700 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:41.704548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-b9db1969-ad78-4390-aa1d-0f75e6824d5a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:41.705682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b9db1969-ad78-4390-aa1d-0f75e6824d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:41.803687 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 666/1342] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:42.089348 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f1736-a4a9-4fc4-a99b-9d0edb217221 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:42.090917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d10f1736-a4a9-4fc4-a99b-9d0edb217221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:42.141243 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:42.199945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f1736-a4a9-4fc4-a99b-9d0edb217221 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:42.275661 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdb360-b953-4825-964f-ae8febe88c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:42.276277 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdb360-b953-4825-964f-ae8febe88c98 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:42.276554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdb360-b953-4825-964f-ae8febe88c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:42.276824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdb360-b953-4825-964f-ae8febe88c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:42.278461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5fcdb360-b953-4825-964f-ae8febe88c98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:42.374866 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:42.392642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcdb360-b953-4825-964f-ae8febe88c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:42.394072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcdb360-b953-4825-964f-ae8febe88c98 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:42.413262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 677/1343] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:45.174458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28d50d-a050-49f8-a5fa-2257900f108b None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:45.175044 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28d50d-a050-49f8-a5fa-2257900f108b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:45.175044 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28d50d-a050-49f8-a5fa-2257900f108b None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:45.176354 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 667/1344] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:45.181821 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3cf9c-4025-40f6-953b-1fb64ec97f22 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:45.182284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3cf9c-4025-40f6-953b-1fb64ec97f22 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:45.182533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3cf9c-4025-40f6-953b-1fb64ec97f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:45.256748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c3f3cf9c-4025-40f6-953b-1fb64ec97f22 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:45.257951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c3f3cf9c-4025-40f6-953b-1fb64ec97f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:45.355237 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 678/1345] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:45.638332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28d50d-a050-49f8-a5fa-2257900f108b None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:45.639875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2c28d50d-a050-49f8-a5fa-2257900f108b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:45.737376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:45.756723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28d50d-a050-49f8-a5fa-2257900f108b None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:45.834010 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381ea40-ac4e-4475-8d8e-6ee010e66504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:45.835229 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381ea40-ac4e-4475-8d8e-6ee010e66504 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:45.835507 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381ea40-ac4e-4475-8d8e-6ee010e66504 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:45.835776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381ea40-ac4e-4475-8d8e-6ee010e66504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:45.837392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6381ea40-ac4e-4475-8d8e-6ee010e66504 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:45.949976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381ea40-ac4e-4475-8d8e-6ee010e66504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:45.951355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381ea40-ac4e-4475-8d8e-6ee010e66504 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:45.975273 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 668/1346] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:50.177455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b023ad-e846-4f32-b940-0bf2a1684d83 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:50.178552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b023ad-e846-4f32-b940-0bf2a1684d83 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:50.178706 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b023ad-e846-4f32-b940-0bf2a1684d83 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:50.182196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 679/1347] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:50.185649 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d28210-8091-42af-afa1-7f781aa2d849 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:50.186283 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d28210-8091-42af-afa1-7f781aa2d849 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:50.186283 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d28210-8091-42af-afa1-7f781aa2d849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:50.207663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:50.278451 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f6d28210-8091-42af-afa1-7f781aa2d849 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:50.280402 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f6d28210-8091-42af-afa1-7f781aa2d849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:50.391345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 669/1348] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:58:50.676775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b023ad-e846-4f32-b940-0bf2a1684d83 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:50.678450 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-65b023ad-e846-4f32-b940-0bf2a1684d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:50.735963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:50.783701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b023ad-e846-4f32-b940-0bf2a1684d83 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:50.859859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08abd1-6494-4d5d-b455-7b4a7adad30a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:50.860472 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08abd1-6494-4d5d-b455-7b4a7adad30a service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:50.860750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08abd1-6494-4d5d-b455-7b4a7adad30a service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:50.861019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08abd1-6494-4d5d-b455-7b4a7adad30a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:50.862656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4c08abd1-6494-4d5d-b455-7b4a7adad30a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:50.967909 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:50.971255 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08abd1-6494-4d5d-b455-7b4a7adad30a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:50.973354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08abd1-6494-4d5d-b455-7b4a7adad30a service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:50.992217 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 680/1349] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:58:55.950859 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cda55-5700-4a97-b020-70a767ed9b64 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:55.951398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cda55-5700-4a97-b020-70a767ed9b64 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:55.951772 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cda55-5700-4a97-b020-70a767ed9b64 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:55.953436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 670/1350] 10.222.0.62 () {54 vars in 921 bytes} [Fri Sep 9 11:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:55.958917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b32dc0-db94-407f-a432-06bbed6f7ddf None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:55.959396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b32dc0-db94-407f-a432-06bbed6f7ddf None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:55.959493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b32dc0-db94-407f-a432-06bbed6f7ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:56.056788 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-00b32dc0-db94-407f-a432-06bbed6f7ddf None None] MFA Rules not processed for user `c58bae3ef7784671ae247f4ad4199c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:56.058229 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-00b32dc0-db94-407f-a432-06bbed6f7ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:56.212682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 681/1351] 10.222.0.62 () {58 vars in 1018 bytes} [Fri Sep 9 11:58:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:00.377142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13bdc0-c719-4370-bbd5-9131f480d4a8 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:00.377690 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13bdc0-c719-4370-bbd5-9131f480d4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:00.377790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13bdc0-c719-4370-bbd5-9131f480d4a8 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:00.380086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 671/1352] 10.222.0.62 () {54 vars in 921 bytes} [Fri Sep 9 11:59:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:00.386155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba0a0f-c246-4a83-812d-5e98fb65a850 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:00.386618 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba0a0f-c246-4a83-812d-5e98fb65a850 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:00.386698 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba0a0f-c246-4a83-812d-5e98fb65a850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:00.390479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=59 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:00.445579 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:00.470595 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-bfba0a0f-c246-4a83-812d-5e98fb65a850 None None] MFA Rules not processed for user `c58bae3ef7784671ae247f4ad4199c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:00.472084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bfba0a0f-c246-4a83-812d-5e98fb65a850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:00.585028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 682/1353] 10.222.0.62 () {58 vars in 1018 bytes} [Fri Sep 9 11:59:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:02.002321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33d4a5-7142-44b0-923f-33981b79baa5 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:02.002818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33d4a5-7142-44b0-923f-33981b79baa5 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:02.002933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33d4a5-7142-44b0-923f-33981b79baa5 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:02.004745 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 672/1354] 10.222.0.62 () {54 vars in 921 bytes} [Fri Sep 9 11:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:02.010012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56f5d3-85bb-43a9-b309-d8a47a57f99d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:02.010989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56f5d3-85bb-43a9-b309-d8a47a57f99d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:02.011238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56f5d3-85bb-43a9-b309-d8a47a57f99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:02.089246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9d56f5d3-85bb-43a9-b309-d8a47a57f99d None None] MFA Rules not processed for user `c58bae3ef7784671ae247f4ad4199c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:02.090517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9d56f5d3-85bb-43a9-b309-d8a47a57f99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:02.133770 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:02.210535 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 683/1355] 10.222.0.62 () {58 vars in 1018 bytes} [Fri Sep 9 11:59:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:05.749814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3c39b-7d53-41d3-a732-f49545c79d16 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:05.751196 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3c39b-7d53-41d3-a732-f49545c79d16 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:05.751294 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3c39b-7d53-41d3-a732-f49545c79d16 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:05.752661 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 673/1356] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:59:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:05.758569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bd94d-444e-4446-ad68-dacf984bb9a5 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:05.759389 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bd94d-444e-4446-ad68-dacf984bb9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:05.759815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bd94d-444e-4446-ad68-dacf984bb9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:05.850351 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5e4bd94d-444e-4446-ad68-dacf984bb9a5 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:05.851690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5e4bd94d-444e-4446-ad68-dacf984bb9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:05.964828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 684/1357] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:06.277435 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31118ae-2b63-456a-8ba6-029f4f02cec0 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:06.278604 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31118ae-2b63-456a-8ba6-029f4f02cec0 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:06.279170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31118ae-2b63-456a-8ba6-029f4f02cec0 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:06.281653 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 674/1358] 10.222.0.62 () {54 vars in 962 bytes} [Fri Sep 9 11:59:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:06.289928 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6d924-7a41-4434-b70e-aba425ea8efc None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:06.290850 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6d924-7a41-4434-b70e-aba425ea8efc None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:06.292557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6d924-7a41-4434-b70e-aba425ea8efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:06.313689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:06.387828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4ae6d924-7a41-4434-b70e-aba425ea8efc None None] MFA Rules not processed for user `1b30e80c421245b5a1ed5849b28ea90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:06.389354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4ae6d924-7a41-4434-b70e-aba425ea8efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:06.526388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 685/1359] 10.222.0.62 () {58 vars in 1059 bytes} [Fri Sep 9 11:59:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:59:06.534071 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31118ae-2b63-456a-8ba6-029f4f02cec0 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:06.536095 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e31118ae-2b63-456a-8ba6-029f4f02cec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:06.712058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31118ae-2b63-456a-8ba6-029f4f02cec0 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:06.809696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:06.856680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d261a-4f10-42d4-bc4d-dbd793fd2277 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:06.857196 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d261a-4f10-42d4-bc4d-dbd793fd2277 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:06.857514 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d261a-4f10-42d4-bc4d-dbd793fd2277 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:06.857738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d261a-4f10-42d4-bc4d-dbd793fd2277 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:06.859570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7e7d261a-4f10-42d4-bc4d-dbd793fd2277 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:06.975740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d261a-4f10-42d4-bc4d-dbd793fd2277 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:06.977569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d261a-4f10-42d4-bc4d-dbd793fd2277 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:06.999734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 675/1360] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:09.398916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f85710-a9b1-4d94-a288-cd6c419853c2 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:09.399621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f85710-a9b1-4d94-a288-cd6c419853c2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:09.399621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f85710-a9b1-4d94-a288-cd6c419853c2 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:09.406159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed804c-a4e3-4237-a066-17095999160a None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:09.406159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed804c-a4e3-4237-a066-17095999160a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:09.406939 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed804c-a4e3-4237-a066-17095999160a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:09.411217 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 686/1361] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:59:09 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:09.500333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-62ed804c-a4e3-4237-a066-17095999160a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:09.501618 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-62ed804c-a4e3-4237-a066-17095999160a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:09.546315 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:09.625320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 676/1362] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:09.925625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f85710-a9b1-4d94-a288-cd6c419853c2 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:09.927742 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-70f85710-a9b1-4d94-a288-cd6c419853c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:10.004953 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:10.116871 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f85710-a9b1-4d94-a288-cd6c419853c2 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:10.263139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945b1c8-4c43-47e2-8cf1-7899060b3c87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:10.264019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945b1c8-4c43-47e2-8cf1-7899060b3c87 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:10.264297 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945b1c8-4c43-47e2-8cf1-7899060b3c87 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:10.264570 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945b1c8-4c43-47e2-8cf1-7899060b3c87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:10.266272 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8945b1c8-4c43-47e2-8cf1-7899060b3c87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:10.291399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:10.368107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945b1c8-4c43-47e2-8cf1-7899060b3c87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:10.370884 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945b1c8-4c43-47e2-8cf1-7899060b3c87 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:10.393747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 687/1363] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:12.972786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecac03b-d9fe-427a-ac32-b60e2695d4e2 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:12.973374 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecac03b-d9fe-427a-ac32-b60e2695d4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:12.973374 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecac03b-d9fe-427a-ac32-b60e2695d4e2 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:12.974929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 677/1364] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:12.980649 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2ee0c-c8ad-4ec8-8725-6579e21283b3 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:12.981094 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2ee0c-c8ad-4ec8-8725-6579e21283b3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:12.981176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2ee0c-c8ad-4ec8-8725-6579e21283b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:13.053853 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-add2ee0c-c8ad-4ec8-8725-6579e21283b3 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:13.055250 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-add2ee0c-c8ad-4ec8-8725-6579e21283b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:13.162832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:13.174439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 688/1365] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:13.489565 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be0bf1-6004-4cc5-b617-fb69ca2b1c70 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:13.489910 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be0bf1-6004-4cc5-b617-fb69ca2b1c70 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:13.490392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be0bf1-6004-4cc5-b617-fb69ca2b1c70 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:13.492439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 678/1366] 10.222.0.62 () {54 vars in 962 bytes} [Fri Sep 9 11:59:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:13.499459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3665722-8262-4c1e-a006-8510ef0dedf5 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:13.500066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3665722-8262-4c1e-a006-8510ef0dedf5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:13.500647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3665722-8262-4c1e-a006-8510ef0dedf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:13.606258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b3665722-8262-4c1e-a006-8510ef0dedf5 None None] MFA Rules not processed for user `1b30e80c421245b5a1ed5849b28ea90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:13.608298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b3665722-8262-4c1e-a006-8510ef0dedf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:13.741288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 689/1367] 10.222.0.62 () {58 vars in 1059 bytes} [Fri Sep 9 11:59:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:13.748985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be0bf1-6004-4cc5-b617-fb69ca2b1c70 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:13.750679 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-00be0bf1-6004-4cc5-b617-fb69ca2b1c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:13.928624 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be0bf1-6004-4cc5-b617-fb69ca2b1c70 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:13.950906 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:14.058862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1398ea-c427-468e-9802-790c5055310a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:14.059516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1398ea-c427-468e-9802-790c5055310a service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:14.059753 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1398ea-c427-468e-9802-790c5055310a service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:14.059923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1398ea-c427-468e-9802-790c5055310a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:14.061485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-3a1398ea-c427-468e-9802-790c5055310a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:14.080844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb4b6-28c2-4f7f-9bef-67461b3310e7 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:14.081414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb4b6-28c2-4f7f-9bef-67461b3310e7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:14.081414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb4b6-28c2-4f7f-9bef-67461b3310e7 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:14.083410 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 690/1368] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:59:14.088458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792615f7-ae6b-48d4-b1a1-113fd96017ff None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:14.088729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792615f7-ae6b-48d4-b1a1-113fd96017ff None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:14.089002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792615f7-ae6b-48d4-b1a1-113fd96017ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:14.154047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:14.161485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1398ea-c427-468e-9802-790c5055310a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:14.163230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1398ea-c427-468e-9802-790c5055310a service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:14.164467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-792615f7-ae6b-48d4-b1a1-113fd96017ff None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:14.165728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-792615f7-ae6b-48d4-b1a1-113fd96017ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:14.183751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 679/1369] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:14.304572 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 691/1370] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:59:14.680087 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bc114-4aac-424b-8508-2f8bd3edf5c6 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:14.680400 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bc114-4aac-424b-8508-2f8bd3edf5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:14.680639 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bc114-4aac-424b-8508-2f8bd3edf5c6 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:14.682264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 680/1371] 10.222.0.62 () {54 vars in 964 bytes} [Fri Sep 9 11:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:14.689956 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a543716-8e24-438b-a089-3e4566f4a64c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:14.690925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a543716-8e24-438b-a089-3e4566f4a64c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:14.691764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a543716-8e24-438b-a089-3e4566f4a64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:14.777431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1a543716-8e24-438b-a089-3e4566f4a64c None None] MFA Rules not processed for user `a25411d51a524f4b9f0ff846a0b29b6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:14.778813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1a543716-8e24-438b-a089-3e4566f4a64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:14.857891 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:14.866359 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 692/1372] 10.222.0.62 () {58 vars in 1061 bytes} [Fri Sep 9 11:59:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:14.875280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bc114-4aac-424b-8508-2f8bd3edf5c6 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:14.876811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-126bc114-4aac-424b-8508-2f8bd3edf5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:14.907086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:14.988810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bc114-4aac-424b-8508-2f8bd3edf5c6 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:15.068813 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b21868-73be-4b11-b690-e4fd700bbab0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:15.069509 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b21868-73be-4b11-b690-e4fd700bbab0 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:15.069774 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b21868-73be-4b11-b690-e4fd700bbab0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:15.069930 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b21868-73be-4b11-b690-e4fd700bbab0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:15.071571 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b0b21868-73be-4b11-b690-e4fd700bbab0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:15.163887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:15.213332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b21868-73be-4b11-b690-e4fd700bbab0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:15.214781 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b21868-73be-4b11-b690-e4fd700bbab0 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:15.244144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 681/1373] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:16.471850 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c79461-ab8a-41da-847a-6d6b410e21bf None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:16.472393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c79461-ab8a-41da-847a-6d6b410e21bf None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:16.472393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c79461-ab8a-41da-847a-6d6b410e21bf None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:16.474352 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 693/1374] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:59:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:16.481872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1ad85-89b8-4517-be25-efe2df0cb416 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:16.482393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1ad85-89b8-4517-be25-efe2df0cb416 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:16.482393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1ad85-89b8-4517-be25-efe2df0cb416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:16.564007 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-1bb1ad85-89b8-4517-be25-efe2df0cb416 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:16.565256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-1bb1ad85-89b8-4517-be25-efe2df0cb416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:16.669980 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 682/1375] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:16.978707 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c79461-ab8a-41da-847a-6d6b410e21bf None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:16.980497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c2c79461-ab8a-41da-847a-6d6b410e21bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:17.141265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c79461-ab8a-41da-847a-6d6b410e21bf None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:17.212041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:17.277119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c50c8-7c1a-4e84-bef4-624c2da23162 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:17.277739 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c50c8-7c1a-4e84-bef4-624c2da23162 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:17.278011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c50c8-7c1a-4e84-bef4-624c2da23162 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:17.278311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c50c8-7c1a-4e84-bef4-624c2da23162 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:17.279927 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fd3c50c8-7c1a-4e84-bef4-624c2da23162 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:17.380701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c50c8-7c1a-4e84-bef4-624c2da23162 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:17.382324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c50c8-7c1a-4e84-bef4-624c2da23162 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:17.401401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 694/1376] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:19.657644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0046eb-1ec7-4450-8f79-73df18a15c83 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:19.658260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0046eb-1ec7-4450-8f79-73df18a15c83 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:19.658491 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0046eb-1ec7-4450-8f79-73df18a15c83 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:19.660118 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 683/1377] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:19.665474 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21846b-8f5f-4973-8abe-cc1f17b632bd None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:19.665931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21846b-8f5f-4973-8abe-cc1f17b632bd None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:19.666033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21846b-8f5f-4973-8abe-cc1f17b632bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:19.736225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2f21846b-8f5f-4973-8abe-cc1f17b632bd None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:19.737548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2f21846b-8f5f-4973-8abe-cc1f17b632bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:19.768760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:19.850152 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 695/1378] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:20.134710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0046eb-1ec7-4450-8f79-73df18a15c83 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:20.136402 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-5c0046eb-1ec7-4450-8f79-73df18a15c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:20.176128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:20.304644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0046eb-1ec7-4450-8f79-73df18a15c83 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:20.426799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:20.433785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2accd4-c5bc-46ae-b57c-25b8677e5b75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:20.434637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2accd4-c5bc-46ae-b57c-25b8677e5b75 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:20.435008 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2accd4-c5bc-46ae-b57c-25b8677e5b75 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:20.435370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2accd4-c5bc-46ae-b57c-25b8677e5b75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:20.437562 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ef2accd4-c5bc-46ae-b57c-25b8677e5b75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:20.538391 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2accd4-c5bc-46ae-b57c-25b8677e5b75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:20.540147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2accd4-c5bc-46ae-b57c-25b8677e5b75 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:20.559793 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 684/1379] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:22.796484 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e28e3-30cc-45ac-82bd-b25d483774e5 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:22.796956 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e28e3-30cc-45ac-82bd-b25d483774e5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:22.797144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e28e3-30cc-45ac-82bd-b25d483774e5 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:22.802493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 696/1380] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:59:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:22.803772 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91845e8-8ae3-4ed2-a628-b418dbe11ca1 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:22.804166 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91845e8-8ae3-4ed2-a628-b418dbe11ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:22.804344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91845e8-8ae3-4ed2-a628-b418dbe11ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:22.915091 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-e91845e8-8ae3-4ed2-a628-b418dbe11ca1 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:22.916303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e91845e8-8ae3-4ed2-a628-b418dbe11ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:22.993081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:23.029038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 685/1381] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:59:23.333696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e28e3-30cc-45ac-82bd-b25d483774e5 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:23.335305 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cc0e28e3-30cc-45ac-82bd-b25d483774e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:23.511546 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e28e3-30cc-45ac-82bd-b25d483774e5 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:23.528490 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:23.656772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad428e3-cf64-4f1a-bc62-298058a9a6e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:23.657762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad428e3-cf64-4f1a-bc62-298058a9a6e4 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:23.658144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad428e3-cf64-4f1a-bc62-298058a9a6e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:23.658776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad428e3-cf64-4f1a-bc62-298058a9a6e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:23.661090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3ad428e3-cf64-4f1a-bc62-298058a9a6e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:23.783085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad428e3-cf64-4f1a-bc62-298058a9a6e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:23.785938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad428e3-cf64-4f1a-bc62-298058a9a6e4 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:23.816975 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 697/1382] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:26.349744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7f090-339f-4a05-82c4-541dcf2b7d7f None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:26.350308 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7f090-339f-4a05-82c4-541dcf2b7d7f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:26.350527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7f090-339f-4a05-82c4-541dcf2b7d7f None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:26.352152 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 686/1383] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:59:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:26.359094 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47bb66-0080-42e4-9605-83134f16c157 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:26.359548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47bb66-0080-42e4-9605-83134f16c157 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:26.359744 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47bb66-0080-42e4-9605-83134f16c157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:26.386332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:26.438399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-bb47bb66-0080-42e4-9605-83134f16c157 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:26.439786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bb47bb66-0080-42e4-9605-83134f16c157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:26.556742 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 698/1384] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:26.848554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7f090-339f-4a05-82c4-541dcf2b7d7f None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:26.850138 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ccc7f090-339f-4a05-82c4-541dcf2b7d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:27.030912 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7f090-339f-4a05-82c4-541dcf2b7d7f None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:27.094902 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:27.170350 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679152ad-8173-4f68-b9de-484946651c2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:27.171119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679152ad-8173-4f68-b9de-484946651c2f service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:27.171407 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679152ad-8173-4f68-b9de-484946651c2f service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:27.171684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679152ad-8173-4f68-b9de-484946651c2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:27.173600 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-679152ad-8173-4f68-b9de-484946651c2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:27.298354 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679152ad-8173-4f68-b9de-484946651c2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:27.300713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679152ad-8173-4f68-b9de-484946651c2f service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:27.323902 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 687/1385] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:29.654576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef07a4-05f8-4177-80dc-1df52415da9c None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:29.655183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef07a4-05f8-4177-80dc-1df52415da9c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:29.655183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef07a4-05f8-4177-80dc-1df52415da9c None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:29.656544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 699/1386] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:29.662523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f75520-510a-4378-b837-38f582ed360f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:29.662950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f75520-510a-4378-b837-38f582ed360f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:29.663072 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f75520-510a-4378-b837-38f582ed360f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:29.737023 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-65f75520-510a-4378-b837-38f582ed360f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:29.738258 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-65f75520-510a-4378-b837-38f582ed360f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:29.746894 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:29.849007 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 688/1387] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:30.141100 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef07a4-05f8-4177-80dc-1df52415da9c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:30.142652 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-68ef07a4-05f8-4177-80dc-1df52415da9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:30.246683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:30.304158 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef07a4-05f8-4177-80dc-1df52415da9c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:30.421285 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20d41e-1e62-4d38-bfea-2860a0c92e11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:30.421948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20d41e-1e62-4d38-bfea-2860a0c92e11 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:30.422267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20d41e-1e62-4d38-bfea-2860a0c92e11 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:30.422541 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20d41e-1e62-4d38-bfea-2860a0c92e11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:30.424140 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1d20d41e-1e62-4d38-bfea-2860a0c92e11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:30.487502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:30.522468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20d41e-1e62-4d38-bfea-2860a0c92e11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:30.524112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20d41e-1e62-4d38-bfea-2860a0c92e11 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:30.544858 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 700/1388] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:30.633817 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d852de-440a-4063-afe7-fd1571225752 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:30.634102 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d852de-440a-4063-afe7-fd1571225752 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:30.634416 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d852de-440a-4063-afe7-fd1571225752 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:30.635807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 689/1389] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:30.642641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932cbb6-db9f-4a57-98ab-e565da81b4b9 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:30.642931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932cbb6-db9f-4a57-98ab-e565da81b4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:30.643200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932cbb6-db9f-4a57-98ab-e565da81b4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:30.720537 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b932cbb6-db9f-4a57-98ab-e565da81b4b9 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:30.721689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b932cbb6-db9f-4a57-98ab-e565da81b4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:30.831300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 701/1390] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:31.129622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e4e15-107f-40a6-9c84-de1d653d5878 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:31.130104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e4e15-107f-40a6-9c84-de1d653d5878 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:31.130821 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e4e15-107f-40a6-9c84-de1d653d5878 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:31.132094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 690/1391] 10.222.0.62 () {54 vars in 966 bytes} [Fri Sep 9 11:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:31.137474 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a65bd2-7316-4acb-91f8-a07f321468f7 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:31.137835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a65bd2-7316-4acb-91f8-a07f321468f7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:31.138107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a65bd2-7316-4acb-91f8-a07f321468f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:31.303676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:31.337447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-10a65bd2-7316-4acb-91f8-a07f321468f7 None None] MFA Rules not processed for user `2d477a49a1724b7e8a4699a5ebd04bde`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:31.338688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-10a65bd2-7316-4acb-91f8-a07f321468f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:31.420206 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 702/1392] 10.222.0.62 () {58 vars in 1063 bytes} [Fri Sep 9 11:59:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:31.429703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e4e15-107f-40a6-9c84-de1d653d5878 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:31.431250 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-024e4e15-107f-40a6-9c84-de1d653d5878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:31.538144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e4e15-107f-40a6-9c84-de1d653d5878 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:31.576053 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:31.612883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613b466-6e0b-492c-ba05-2c5c0a5e0d1b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d477a49a1724b7e8a4699a5ebd04bde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:31.613394 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613b466-6e0b-492c-ba05-2c5c0a5e0d1b service glance] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:31.613668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613b466-6e0b-492c-ba05-2c5c0a5e0d1b service glance] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:31.613941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613b466-6e0b-492c-ba05-2c5c0a5e0d1b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:31.615608 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f613b466-6e0b-492c-ba05-2c5c0a5e0d1b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:31.728751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613b466-6e0b-492c-ba05-2c5c0a5e0d1b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:31.730471 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613b466-6e0b-492c-ba05-2c5c0a5e0d1b service glance] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:31.748903 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 691/1393] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:32.496205 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61843e-c540-4035-b985-f03300f495e1 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.496708 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61843e-c540-4035-b985-f03300f495e1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.496792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61843e-c540-4035-b985-f03300f495e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.572263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1b61843e-c540-4035-b985-f03300f495e1 None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:32.573469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1b61843e-c540-4035-b985-f03300f495e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:32.659685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:32.667875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 703/1394] 10.222.0.62 () {58 vars in 1009 bytes} [Fri Sep 9 11:59:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:32.726169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ee67a-f337-4aa9-bf74-958370df5cf4 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.726788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ee67a-f337-4aa9-bf74-958370df5cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.727018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ee67a-f337-4aa9-bf74-958370df5cf4 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.728469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 692/1395] 10.222.0.62 () {54 vars in 943 bytes} [Fri Sep 9 11:59:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:32.736408 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5d702-a446-446f-a4ef-bfd0f96081a9 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.736639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5d702-a446-446f-a4ef-bfd0f96081a9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.736933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5d702-a446-446f-a4ef-bfd0f96081a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.816975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e799570-528c-487f-a381-1f9e45e7c15e None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.817321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e799570-528c-487f-a381-1f9e45e7c15e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.817496 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e799570-528c-487f-a381-1f9e45e7c15e None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.819704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 693/1396] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:59:32.824761 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e4b5d702-a446-446f-a4ef-bfd0f96081a9 None None] MFA Rules not processed for user `1b2de52919e54bf9bcc352909c9f905f`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:32.826105 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5223f02-e220-4f02-97de-d2f8ce44b5e2 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.826379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e4b5d702-a446-446f-a4ef-bfd0f96081a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:32.826539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5223f02-e220-4f02-97de-d2f8ce44b5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.826670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5223f02-e220-4f02-97de-d2f8ce44b5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.900295 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-c5223f02-e220-4f02-97de-d2f8ce44b5e2 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:32.901831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-c5223f02-e220-4f02-97de-d2f8ce44b5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:32.921755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:32.942026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 704/1397] 10.222.0.62 () {58 vars in 1040 bytes} [Fri Sep 9 11:59:32 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:32.952858 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5d702-a446-446f-a4ef-bfd0f96081a9 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:32.964630 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5d702-a446-446f-a4ef-bfd0f96081a9 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:33.015468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 694/1398] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 11:59:33.033961 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:33.104132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe8a9b-a536-40df-be53-5c8b4f1b774d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:33.104744 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe8a9b-a536-40df-be53-5c8b4f1b774d service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.105018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe8a9b-a536-40df-be53-5c8b4f1b774d service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.105538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe8a9b-a536-40df-be53-5c8b4f1b774d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.107213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cbbe8a9b-a536-40df-be53-5c8b4f1b774d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:33.199021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe8a9b-a536-40df-be53-5c8b4f1b774d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:33.200633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe8a9b-a536-40df-be53-5c8b4f1b774d service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:33.209911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 705/1399] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 11:59:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 11:59:33.334732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5223f02-e220-4f02-97de-d2f8ce44b5e2 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:33.335924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-c5223f02-e220-4f02-97de-d2f8ce44b5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:33.470249 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5223f02-e220-4f02-97de-d2f8ce44b5e2 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:33.511833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:33.605607 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4a465-be69-478c-b86c-2a7afb31ced7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:33.606272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4a465-be69-478c-b86c-2a7afb31ced7 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.606552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4a465-be69-478c-b86c-2a7afb31ced7 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.606822 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4a465-be69-478c-b86c-2a7afb31ced7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.610934 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-3cf4a465-be69-478c-b86c-2a7afb31ced7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:33.716287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4a465-be69-478c-b86c-2a7afb31ced7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:33.717943 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4a465-be69-478c-b86c-2a7afb31ced7 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:33.757142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 695/1400] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:35.985492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5527c-804a-4fc4-85e2-e56b01676732 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.986088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5527c-804a-4fc4-85e2-e56b01676732 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.986088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5527c-804a-4fc4-85e2-e56b01676732 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.987578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 706/1401] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:35.992969 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea666b-4af0-4a68-9864-6b19fc5b440d None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.993446 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea666b-4af0-4a68-9864-6b19fc5b440d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.993547 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea666b-4af0-4a68-9864-6b19fc5b440d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:36.049590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:36.072085 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-00ea666b-4af0-4a68-9864-6b19fc5b440d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:36.073310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-00ea666b-4af0-4a68-9864-6b19fc5b440d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:36.196162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 696/1402] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:36.495250 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5527c-804a-4fc4-85e2-e56b01676732 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:36.496811 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-40b5527c-804a-4fc4-85e2-e56b01676732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:36.684000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:36.777509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5527c-804a-4fc4-85e2-e56b01676732 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:36.907228 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61abf8-c07d-4214-853a-33f789dab673 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:36.907932 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61abf8-c07d-4214-853a-33f789dab673 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:36.908343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61abf8-c07d-4214-853a-33f789dab673 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:36.908614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61abf8-c07d-4214-853a-33f789dab673 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:36.910241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2e61abf8-c07d-4214-853a-33f789dab673 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:36.955657 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:37.013300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61abf8-c07d-4214-853a-33f789dab673 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:37.014926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61abf8-c07d-4214-853a-33f789dab673 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:37.033845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 707/1403] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:39.346273 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07aec76-20bd-437d-8609-159a8abb5b6b None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:39.346866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07aec76-20bd-437d-8609-159a8abb5b6b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:39.346866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07aec76-20bd-437d-8609-159a8abb5b6b None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:39.348355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 697/1404] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:39.354149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10555d2-9f26-4ff1-b225-5170a42b1f8b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:39.354606 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10555d2-9f26-4ff1-b225-5170a42b1f8b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:39.354691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10555d2-9f26-4ff1-b225-5170a42b1f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:39.432472 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f10555d2-9f26-4ff1-b225-5170a42b1f8b None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:39.433703 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f10555d2-9f26-4ff1-b225-5170a42b1f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:39.552333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 708/1405] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:39.844621 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07aec76-20bd-437d-8609-159a8abb5b6b None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:39.846155 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a07aec76-20bd-437d-8609-159a8abb5b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:39.990979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:40.028638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07aec76-20bd-437d-8609-159a8abb5b6b None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:40.165832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7facc2c-ab88-4375-a0c7-cd4a2db5ecd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:40.166366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7facc2c-ab88-4375-a0c7-cd4a2db5ecd3 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.166644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7facc2c-ab88-4375-a0c7-cd4a2db5ecd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.166914 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7facc2c-ab88-4375-a0c7-cd4a2db5ecd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.168535 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a7facc2c-ab88-4375-a0c7-cd4a2db5ecd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:40.268700 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:40.280653 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7facc2c-ab88-4375-a0c7-cd4a2db5ecd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:40.282312 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7facc2c-ab88-4375-a0c7-cd4a2db5ecd3 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:40.305324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 698/1406] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 11:59:42.513798 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af8975-f0c6-48f0-a444-6b6f86420628 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:42.514386 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af8975-f0c6-48f0-a444-6b6f86420628 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:42.514386 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af8975-f0c6-48f0-a444-6b6f86420628 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:42.516060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 709/1407] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 11:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:42.522908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c229e8-bcca-4ddb-b091-4f068596635c None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:42.523404 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c229e8-bcca-4ddb-b091-4f068596635c None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:42.523506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c229e8-bcca-4ddb-b091-4f068596635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:42.598857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-82c229e8-bcca-4ddb-b091-4f068596635c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:42.600086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-82c229e8-bcca-4ddb-b091-4f068596635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:42.709325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 699/1408] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:43.014279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af8975-f0c6-48f0-a444-6b6f86420628 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:43.016035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a6af8975-f0c6-48f0-a444-6b6f86420628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:43.038170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:43.187252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af8975-f0c6-48f0-a444-6b6f86420628 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:43.295036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:43.322268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010e92a-232d-44eb-a07c-a0a2738236aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:43.322883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010e92a-232d-44eb-a07c-a0a2738236aa service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:43.323156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010e92a-232d-44eb-a07c-a0a2738236aa service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:43.323422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010e92a-232d-44eb-a07c-a0a2738236aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:43.325030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0010e92a-232d-44eb-a07c-a0a2738236aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:43.430336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010e92a-232d-44eb-a07c-a0a2738236aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:43.432134 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010e92a-232d-44eb-a07c-a0a2738236aa service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:43.452724 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 710/1409] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 11:59:52.707764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=74 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:53.450127 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=54 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:56.187868 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff7d62-410d-43eb-a79e-9d51e3a91086 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:56.188434 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff7d62-410d-43eb-a79e-9d51e3a91086 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:56.188434 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff7d62-410d-43eb-a79e-9d51e3a91086 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:56.189683 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 700/1410] 10.222.0.62 () {54 vars in 918 bytes} [Fri Sep 9 11:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:56.195061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a363665-b66b-4a0c-b7ca-8232feb95e55 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:56.195543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a363665-b66b-4a0c-b7ca-8232feb95e55 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:56.195643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a363665-b66b-4a0c-b7ca-8232feb95e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:56.271873 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5a363665-b66b-4a0c-b7ca-8232feb95e55 None None] MFA Rules not processed for user `c58bae3ef7784671ae247f4ad4199c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:56.273595 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5a363665-b66b-4a0c-b7ca-8232feb95e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:56.378923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 711/1411] 10.222.0.62 () {58 vars in 1015 bytes} [Fri Sep 9 11:59:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:56.393483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571568f-6361-40d3-86b7-4f33cf64d0e1 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:56.393762 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571568f-6361-40d3-86b7-4f33cf64d0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:56.394032 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571568f-6361-40d3-86b7-4f33cf64d0e1 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:56.395383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 701/1412] 10.222.0.62 () {54 vars in 918 bytes} [Fri Sep 9 11:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:56.400660 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685259b-6016-43a5-a15f-244a398843c4 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:56.400949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685259b-6016-43a5-a15f-244a398843c4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:56.401226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685259b-6016-43a5-a15f-244a398843c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:56.474302 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:56.477021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-0685259b-6016-43a5-a15f-244a398843c4 None None] MFA Rules not processed for user `1b30e80c421245b5a1ed5849b28ea90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:56.478742 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0685259b-6016-43a5-a15f-244a398843c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:56.595404 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 712/1413] 10.222.0.62 () {58 vars in 1015 bytes} [Fri Sep 9 11:59:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:59.179131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144d208-f05b-4b0c-b01c-57730a17c86a None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:59.179732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144d208-f05b-4b0c-b01c-57730a17c86a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:59.179732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144d208-f05b-4b0c-b01c-57730a17c86a None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:59.181017 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 11:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:59.186865 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea953876-79cb-4736-9f64-e4c5012b321c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:59.187314 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea953876-79cb-4736-9f64-e4c5012b321c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:59.187428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea953876-79cb-4736-9f64-e4c5012b321c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:59.255001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ea953876-79cb-4736-9f64-e4c5012b321c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:59.256491 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ea953876-79cb-4736-9f64-e4c5012b321c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:59.310830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:59.352534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 11:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 11:59:59.644908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144d208-f05b-4b0c-b01c-57730a17c86a None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:59.647217 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-9144d208-f05b-4b0c-b01c-57730a17c86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:59.797190 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144d208-f05b-4b0c-b01c-57730a17c86a None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:59.871808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:59.913148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1e8fe-c350-4745-9441-d8df9331e0f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:59.913662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1e8fe-c350-4745-9441-d8df9331e0f7 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:59.913938 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1e8fe-c350-4745-9441-d8df9331e0f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:59.914241 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1e8fe-c350-4745-9441-d8df9331e0f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:59.915857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b0c1e8fe-c350-4745-9441-d8df9331e0f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:00.006058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1e8fe-c350-4745-9441-d8df9331e0f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:00.008236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1e8fe-c350-4745-9441-d8df9331e0f7 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:00.025916 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 703/1416] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 11:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:01.168644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bff3f-b3e1-4fae-9702-d1f6b8a900e7 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.169220 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bff3f-b3e1-4fae-9702-d1f6b8a900e7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.169220 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bff3f-b3e1-4fae-9702-d1f6b8a900e7 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.170614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:01.176931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9b6e6-b78b-4a0e-8b67-0fd41f9a53de None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.177399 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9b6e6-b78b-4a0e-8b67-0fd41f9a53de None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.177504 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9b6e6-b78b-4a0e-8b67-0fd41f9a53de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.255026 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-9dd9b6e6-b78b-4a0e-8b67-0fd41f9a53de None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:01.256238 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-9dd9b6e6-b78b-4a0e-8b67-0fd41f9a53de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:01.284047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:01.353311 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:01.505396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bff3f-b3e1-4fae-9702-d1f6b8a900e7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:01.507681 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f55bff3f-b3e1-4fae-9702-d1f6b8a900e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:01.652503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bff3f-b3e1-4fae-9702-d1f6b8a900e7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:01.694271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:01.768538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda15bf-a65b-406f-8ac7-1a9685f8909b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:01.769056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda15bf-a65b-406f-8ac7-1a9685f8909b service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.769330 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda15bf-a65b-406f-8ac7-1a9685f8909b service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.769599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda15bf-a65b-406f-8ac7-1a9685f8909b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.771244 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7fda15bf-a65b-406f-8ac7-1a9685f8909b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:01.861145 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda15bf-a65b-406f-8ac7-1a9685f8909b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:01.862773 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda15bf-a65b-406f-8ac7-1a9685f8909b service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:01.884191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 715/1419] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:03.034078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fffc02-423b-4f6a-abc2-debf626deee8 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:03.034641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fffc02-423b-4f6a-abc2-debf626deee8 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:03.034641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fffc02-423b-4f6a-abc2-debf626deee8 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:03.036012 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:03.042798 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795dde5-4fcb-4d51-89b3-fb203b14d48e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:03.043284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795dde5-4fcb-4d51-89b3-fb203b14d48e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:03.043381 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795dde5-4fcb-4d51-89b3-fb203b14d48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:03.097170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:03.120676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-0795dde5-4fcb-4d51-89b3-fb203b14d48e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:03.121885 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0795dde5-4fcb-4d51-89b3-fb203b14d48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:03.216508 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:03.383636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fffc02-423b-4f6a-abc2-debf626deee8 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:03.385173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-c9fffc02-423b-4f6a-abc2-debf626deee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:03.532107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fffc02-423b-4f6a-abc2-debf626deee8 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:03.544858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:03.655944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c76bb6-4a19-47fe-b14d-0acef9ab10b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:03.656561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c76bb6-4a19-47fe-b14d-0acef9ab10b4 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:03.656846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c76bb6-4a19-47fe-b14d-0acef9ab10b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:03.657115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c76bb6-4a19-47fe-b14d-0acef9ab10b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:03.658796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d0c76bb6-4a19-47fe-b14d-0acef9ab10b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:03.748666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c76bb6-4a19-47fe-b14d-0acef9ab10b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:03.750279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c76bb6-4a19-47fe-b14d-0acef9ab10b4 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:03.768107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 706/1422] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:04.905141 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c92b11-e337-4070-a7b0-54ce4b797072 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:04.905808 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c92b11-e337-4070-a7b0-54ce4b797072 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:04.906050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c92b11-e337-4070-a7b0-54ce4b797072 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:04.907455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:00:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:04.915035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943516ad-ddba-4aa6-9b20-b5cb11dafcf3 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:04.915581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943516ad-ddba-4aa6-9b20-b5cb11dafcf3 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:04.915581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943516ad-ddba-4aa6-9b20-b5cb11dafcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:04.935260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:04.996721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-943516ad-ddba-4aa6-9b20-b5cb11dafcf3 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:04.997970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-943516ad-ddba-4aa6-9b20-b5cb11dafcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:05.094880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:05.246441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c92b11-e337-4070-a7b0-54ce4b797072 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:05.247957 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-30c92b11-e337-4070-a7b0-54ce4b797072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:05.361675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:05.398427 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c92b11-e337-4070-a7b0-54ce4b797072 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:05.512018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457cb866-a19c-4e2f-93cd-19c3c5006b84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:05.512646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cb866-a19c-4e2f-93cd-19c3c5006b84 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:05.512902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cb866-a19c-4e2f-93cd-19c3c5006b84 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:05.513168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cb866-a19c-4e2f-93cd-19c3c5006b84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:05.514796 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-457cb866-a19c-4e2f-93cd-19c3c5006b84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:05.593744 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:05.610336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457cb866-a19c-4e2f-93cd-19c3c5006b84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:05.611938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457cb866-a19c-4e2f-93cd-19c3c5006b84 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:05.632273 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 718/1425] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:00:06.755192 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef820d7c-b0dc-4157-beee-a544d067732a None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:06.755760 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef820d7c-b0dc-4157-beee-a544d067732a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:06.755760 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef820d7c-b0dc-4157-beee-a544d067732a None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:06.757090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:06.764088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b075fa-67f7-4b71-97a7-0de08dfb154d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:06.764563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b075fa-67f7-4b71-97a7-0de08dfb154d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:06.764671 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b075fa-67f7-4b71-97a7-0de08dfb154d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:06.839812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a6b075fa-67f7-4b71-97a7-0de08dfb154d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:06.841002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a6b075fa-67f7-4b71-97a7-0de08dfb154d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:06.936618 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:07.088213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef820d7c-b0dc-4157-beee-a544d067732a None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:07.089762 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ef820d7c-b0dc-4157-beee-a544d067732a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:07.176392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:07.242543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef820d7c-b0dc-4157-beee-a544d067732a None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:07.356445 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f2aaa-7a94-462c-b757-58bb0715edf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:07.356971 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f2aaa-7a94-462c-b757-58bb0715edf5 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:07.357249 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f2aaa-7a94-462c-b757-58bb0715edf5 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:07.357519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f2aaa-7a94-462c-b757-58bb0715edf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:07.359436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-fc8f2aaa-7a94-462c-b757-58bb0715edf5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:07.407952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:07.450066 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f2aaa-7a94-462c-b757-58bb0715edf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:07.451716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f2aaa-7a94-462c-b757-58bb0715edf5 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:07.469941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 709/1428] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:08.557762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4886bb4-b449-429c-bc2b-ed50adde5eae None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:08.558352 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4886bb4-b449-429c-bc2b-ed50adde5eae None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:08.558352 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4886bb4-b449-429c-bc2b-ed50adde5eae None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:08.559618 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:08.566142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd43b4-adb2-46a9-a4cf-afac8776e63d None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:08.566584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd43b4-adb2-46a9-a4cf-afac8776e63d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:08.566709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd43b4-adb2-46a9-a4cf-afac8776e63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:08.647011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-0bbd43b4-adb2-46a9-a4cf-afac8776e63d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:08.648246 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0bbd43b4-adb2-46a9-a4cf-afac8776e63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:08.747651 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:08.895469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4886bb4-b449-429c-bc2b-ed50adde5eae None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:08.896965 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d4886bb4-b449-429c-bc2b-ed50adde5eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:08.949908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:09.047306 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4886bb4-b449-429c-bc2b-ed50adde5eae None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:09.160916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd91e6-e6ee-4c85-a71a-e194e514d5a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:09.161518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd91e6-e6ee-4c85-a71a-e194e514d5a4 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:09.161695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd91e6-e6ee-4c85-a71a-e194e514d5a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:09.161960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd91e6-e6ee-4c85-a71a-e194e514d5a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:09.163568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-43fd91e6-e6ee-4c85-a71a-e194e514d5a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:09.170900 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:09.253860 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd91e6-e6ee-4c85-a71a-e194e514d5a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:09.255459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd91e6-e6ee-4c85-a71a-e194e514d5a4 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:09.272823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 721/1431] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:10.343999 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3c8fc-a025-4ab6-ba9d-fe1bb9c5df94 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.344564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3c8fc-a025-4ab6-ba9d-fe1bb9c5df94 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.344564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3c8fc-a025-4ab6-ba9d-fe1bb9c5df94 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.345895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:10.352587 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12a054-50c8-4b85-b81a-3cade973af5f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.353201 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12a054-50c8-4b85-b81a-3cade973af5f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.353201 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12a054-50c8-4b85-b81a-3cade973af5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.428960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-bc12a054-50c8-4b85-b81a-3cade973af5f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:10.430615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bc12a054-50c8-4b85-b81a-3cade973af5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:10.499967 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:10.529882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:10.679077 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3c8fc-a025-4ab6-ba9d-fe1bb9c5df94 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:10.681201 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f8b3c8fc-a025-4ab6-ba9d-fe1bb9c5df94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:10.698840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:10.828424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3c8fc-a025-4ab6-ba9d-fe1bb9c5df94 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:10.916272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:10.944859 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5c4e0-e8f3-4c8b-9d3f-af76e65b0d4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:10.945384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5c4e0-e8f3-4c8b-9d3f-af76e65b0d4e service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.945660 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5c4e0-e8f3-4c8b-9d3f-af76e65b0d4e service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.945924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5c4e0-e8f3-4c8b-9d3f-af76e65b0d4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.947560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-8ae5c4e0-e8f3-4c8b-9d3f-af76e65b0d4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:11.041480 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5c4e0-e8f3-4c8b-9d3f-af76e65b0d4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:11.043107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5c4e0-e8f3-4c8b-9d3f-af76e65b0d4e service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:11.060565 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 712/1434] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:12.109313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfe421-23a5-4836-b834-8cc99b06fe78 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:12.110009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfe421-23a5-4836-b834-8cc99b06fe78 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:12.110009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfe421-23a5-4836-b834-8cc99b06fe78 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:12.111317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:12.117573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6487ab-d620-4f9d-a47a-7912768530d4 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:12.118025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6487ab-d620-4f9d-a47a-7912768530d4 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:12.118164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6487ab-d620-4f9d-a47a-7912768530d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:12.194998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-2d6487ab-d620-4f9d-a47a-7912768530d4 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:12.196755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2d6487ab-d620-4f9d-a47a-7912768530d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:12.239089 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:12.295042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:12.443698 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfe421-23a5-4836-b834-8cc99b06fe78 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:12.445126 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a4bfe421-23a5-4836-b834-8cc99b06fe78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:12.591402 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfe421-23a5-4836-b834-8cc99b06fe78 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:12.650057 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:12.708583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449810e8-5d6e-4ea8-a235-29f01ce8372e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:12.709109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449810e8-5d6e-4ea8-a235-29f01ce8372e service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:12.709390 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449810e8-5d6e-4ea8-a235-29f01ce8372e service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:12.709658 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449810e8-5d6e-4ea8-a235-29f01ce8372e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:12.711797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-449810e8-5d6e-4ea8-a235-29f01ce8372e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:12.802756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449810e8-5d6e-4ea8-a235-29f01ce8372e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:12.804344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449810e8-5d6e-4ea8-a235-29f01ce8372e service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:12.822558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 724/1437] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:13.890347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6710327-90d4-4f70-8517-ac2693b451c2 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:13.890890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6710327-90d4-4f70-8517-ac2693b451c2 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:13.890890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6710327-90d4-4f70-8517-ac2693b451c2 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:13.892279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:13.898944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6fec6-4e5d-4a95-9847-88a31fc3e0e5 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:13.899383 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6fec6-4e5d-4a95-9847-88a31fc3e0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:13.899525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6fec6-4e5d-4a95-9847-88a31fc3e0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:13.975182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-25b6fec6-4e5d-4a95-9847-88a31fc3e0e5 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:13.976397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-25b6fec6-4e5d-4a95-9847-88a31fc3e0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:13.978552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:14.074059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:14.223947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6710327-90d4-4f70-8517-ac2693b451c2 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:14.225441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d6710327-90d4-4f70-8517-ac2693b451c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:14.371632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6710327-90d4-4f70-8517-ac2693b451c2 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:14.397791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:14.483976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28397c-faa3-435b-9fc3-fe4d7711dd55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:14.484480 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28397c-faa3-435b-9fc3-fe4d7711dd55 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:14.484751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28397c-faa3-435b-9fc3-fe4d7711dd55 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:14.485015 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28397c-faa3-435b-9fc3-fe4d7711dd55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:14.486623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-da28397c-faa3-435b-9fc3-fe4d7711dd55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:14.576298 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28397c-faa3-435b-9fc3-fe4d7711dd55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:14.577935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28397c-faa3-435b-9fc3-fe4d7711dd55 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:14.595861 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 715/1440] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:16.667872 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d5037-9ab4-4a9c-ad56-21a597c5c1e8 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:16.668428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d5037-9ab4-4a9c-ad56-21a597c5c1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:16.668428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d5037-9ab4-4a9c-ad56-21a597c5c1e8 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:16.669766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:16.675121 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389122f-61fd-454b-ac59-4d854ded453f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:16.675564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389122f-61fd-454b-ac59-4d854ded453f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:16.675721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389122f-61fd-454b-ac59-4d854ded453f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:16.717380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:16.744880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-8389122f-61fd-454b-ac59-4d854ded453f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:16.746093 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-8389122f-61fd-454b-ac59-4d854ded453f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:16.847734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:17.138336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d5037-9ab4-4a9c-ad56-21a597c5c1e8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:17.139869 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f09d5037-9ab4-4a9c-ad56-21a597c5c1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:17.244920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d5037-9ab4-4a9c-ad56-21a597c5c1e8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:17.278626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:17.323798 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad7892-b2ac-46e5-aae5-1453daa5ec8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:17.324399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad7892-b2ac-46e5-aae5-1453daa5ec8e service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:17.324674 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad7892-b2ac-46e5-aae5-1453daa5ec8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:17.324930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad7892-b2ac-46e5-aae5-1453daa5ec8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:17.326547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1fad7892-b2ac-46e5-aae5-1453daa5ec8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:17.435251 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad7892-b2ac-46e5-aae5-1453daa5ec8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:17.436604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad7892-b2ac-46e5-aae5-1453daa5ec8e service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:17.454601 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 727/1443] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:19.818453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbbecf-a3f0-4d5f-924a-4c7e4a070158 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:19.819041 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbbecf-a3f0-4d5f-924a-4c7e4a070158 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:19.819041 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbbecf-a3f0-4d5f-924a-4c7e4a070158 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:19.820320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:19.826086 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4876ea-69db-42d9-9972-4f503b6d1b25 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:19.826548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4876ea-69db-42d9-9972-4f503b6d1b25 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:19.826679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4876ea-69db-42d9-9972-4f503b6d1b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:19.894992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-3a4876ea-69db-42d9-9972-4f503b6d1b25 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:19.896245 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3a4876ea-69db-42d9-9972-4f503b6d1b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:19.909319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:19.997480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:20.279168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbbecf-a3f0-4d5f-924a-4c7e4a070158 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:20.280751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b6dbbecf-a3f0-4d5f-924a-4c7e4a070158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:20.387758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbbecf-a3f0-4d5f-924a-4c7e4a070158 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:20.401017 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:20.463722 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8da38d-3ac5-489e-902f-b4b16706b5eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:20.464358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8da38d-3ac5-489e-902f-b4b16706b5eb service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:20.464638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8da38d-3ac5-489e-902f-b4b16706b5eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:20.464909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8da38d-3ac5-489e-902f-b4b16706b5eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:20.466608 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-1b8da38d-3ac5-489e-902f-b4b16706b5eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:20.577624 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8da38d-3ac5-489e-902f-b4b16706b5eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:20.579010 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8da38d-3ac5-489e-902f-b4b16706b5eb service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:20.596682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 718/1446] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:23.095060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d938a58-7947-4181-819f-5d8267e91112 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:23.095640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d938a58-7947-4181-819f-5d8267e91112 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:23.095640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d938a58-7947-4181-819f-5d8267e91112 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:23.097098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:23.102616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111233a-16dc-4296-9d21-8ddcbb37f0d7 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:23.103070 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111233a-16dc-4296-9d21-8ddcbb37f0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:23.103178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111233a-16dc-4296-9d21-8ddcbb37f0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:23.153415 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:23.171538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-1111233a-16dc-4296-9d21-8ddcbb37f0d7 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:23.172748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-1111233a-16dc-4296-9d21-8ddcbb37f0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:23.275179 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:23.565200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d938a58-7947-4181-819f-5d8267e91112 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:23.566799 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4d938a58-7947-4181-819f-5d8267e91112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:23.671207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d938a58-7947-4181-819f-5d8267e91112 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:23.710764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:23.748143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a738154-e265-4a98-a4be-557252b68936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:23.748762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a738154-e265-4a98-a4be-557252b68936 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:23.749046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a738154-e265-4a98-a4be-557252b68936 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:23.749296 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a738154-e265-4a98-a4be-557252b68936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:23.751024 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3a738154-e265-4a98-a4be-557252b68936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:23.866860 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a738154-e265-4a98-a4be-557252b68936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:23.868298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a738154-e265-4a98-a4be-557252b68936 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:23.887309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 730/1449] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:26.196665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ba5b7-7f05-4add-a113-c63e675223aa None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:26.197202 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ba5b7-7f05-4add-a113-c63e675223aa None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:26.197202 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ba5b7-7f05-4add-a113-c63e675223aa None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:26.198475 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:26.204061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913934e-e6a6-413c-8263-0ebc7b1ac217 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:26.204520 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913934e-e6a6-413c-8263-0ebc7b1ac217 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:26.204631 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913934e-e6a6-413c-8263-0ebc7b1ac217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:26.271989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2913934e-e6a6-413c-8263-0ebc7b1ac217 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:26.273224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2913934e-e6a6-413c-8263-0ebc7b1ac217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:26.294618 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:26.371805 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:00:26.666078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ba5b7-7f05-4add-a113-c63e675223aa None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:26.667652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d23ba5b7-7f05-4add-a113-c63e675223aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:26.775444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ba5b7-7f05-4add-a113-c63e675223aa None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:26.798300 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:26.852932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbdd85-78da-47c6-a174-fe0c49f05084 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:26.853573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbdd85-78da-47c6-a174-fe0c49f05084 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:26.853851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbdd85-78da-47c6-a174-fe0c49f05084 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:26.854233 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbdd85-78da-47c6-a174-fe0c49f05084 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:26.855821 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e2fbdd85-78da-47c6-a174-fe0c49f05084 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:26.976295 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbdd85-78da-47c6-a174-fe0c49f05084 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:26.978419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbdd85-78da-47c6-a174-fe0c49f05084 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:26.998788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 721/1452] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:29.296862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41a66a-b7d3-49fa-b2ac-21385740288b None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:29.297426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41a66a-b7d3-49fa-b2ac-21385740288b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:29.297426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41a66a-b7d3-49fa-b2ac-21385740288b None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:29.298832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:29.304388 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c8f0f-aef4-4d6d-8dbc-50e42886e6b2 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:29.304794 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c8f0f-aef4-4d6d-8dbc-50e42886e6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:29.304944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c8f0f-aef4-4d6d-8dbc-50e42886e6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:29.368659 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:29.371306 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-954c8f0f-aef4-4d6d-8dbc-50e42886e6b2 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:29.373007 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-954c8f0f-aef4-4d6d-8dbc-50e42886e6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:29.471318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:29.758358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41a66a-b7d3-49fa-b2ac-21385740288b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:29.759899 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4d41a66a-b7d3-49fa-b2ac-21385740288b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:29.867306 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41a66a-b7d3-49fa-b2ac-21385740288b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:29.919357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:29.944023 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc6fde-08ee-48f2-8ea2-65b530562c69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:29.944636 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc6fde-08ee-48f2-8ea2-65b530562c69 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:29.944915 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc6fde-08ee-48f2-8ea2-65b530562c69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:29.945188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc6fde-08ee-48f2-8ea2-65b530562c69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:29.946830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8acc6fde-08ee-48f2-8ea2-65b530562c69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:30.057683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc6fde-08ee-48f2-8ea2-65b530562c69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:30.059099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc6fde-08ee-48f2-8ea2-65b530562c69 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:30.077295 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 733/1455] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:32.349902 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561dd64-d9fa-48bd-8286-9ca29c8e0316 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:32.350537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561dd64-d9fa-48bd-8286-9ca29c8e0316 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:32.350537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561dd64-d9fa-48bd-8286-9ca29c8e0316 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:32.351778 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:32.357300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272738a-ae70-4d8b-9222-51e7415d5151 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:32.357757 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272738a-ae70-4d8b-9222-51e7415d5151 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:32.357871 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272738a-ae70-4d8b-9222-51e7415d5151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:32.423745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7272738a-ae70-4d8b-9222-51e7415d5151 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:32.424945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7272738a-ae70-4d8b-9222-51e7415d5151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:32.454530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:32.525624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:32.819821 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561dd64-d9fa-48bd-8286-9ca29c8e0316 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:32.822022 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0561dd64-d9fa-48bd-8286-9ca29c8e0316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:32.925532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561dd64-d9fa-48bd-8286-9ca29c8e0316 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:32.954763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:33.008342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3e052-0527-4118-af1d-610ba2a70bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:33.009561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3e052-0527-4118-af1d-610ba2a70bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:33.009843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3e052-0527-4118-af1d-610ba2a70bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:33.010125 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3e052-0527-4118-af1d-610ba2a70bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:33.011815 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-9ee3e052-0527-4118-af1d-610ba2a70bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:33.132232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3e052-0527-4118-af1d-610ba2a70bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:33.133704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3e052-0527-4118-af1d-610ba2a70bd5 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:33.151563 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 724/1458] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:35.481358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e35fc-1864-43c9-b6c1-a50d8547f061 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:35.481958 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e35fc-1864-43c9-b6c1-a50d8547f061 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:35.481958 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e35fc-1864-43c9-b6c1-a50d8547f061 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:35.483254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:35.488988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b057d2-57b4-4654-85cf-9fee210ad449 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:35.489428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b057d2-57b4-4654-85cf-9fee210ad449 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:35.489532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b057d2-57b4-4654-85cf-9fee210ad449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:35.554420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-20b057d2-57b4-4654-85cf-9fee210ad449 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:35.556233 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-20b057d2-57b4-4654-85cf-9fee210ad449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:35.564721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:35.654614 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:35.954740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e35fc-1864-43c9-b6c1-a50d8547f061 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:35.956418 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d84e35fc-1864-43c9-b6c1-a50d8547f061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:36.061281 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e35fc-1864-43c9-b6c1-a50d8547f061 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:36.129481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:36.148589 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d1a50-ef1d-42d7-943f-4d3d54f6659e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:36.149209 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d1a50-ef1d-42d7-943f-4d3d54f6659e service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:36.150033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d1a50-ef1d-42d7-943f-4d3d54f6659e service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:36.150334 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d1a50-ef1d-42d7-943f-4d3d54f6659e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:36.151936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8e8d1a50-ef1d-42d7-943f-4d3d54f6659e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:36.266264 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d1a50-ef1d-42d7-943f-4d3d54f6659e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:36.267688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d1a50-ef1d-42d7-943f-4d3d54f6659e service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:36.286121 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 736/1461] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:38.504387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d983e-6276-4880-a226-0b846427a4b6 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:38.504935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d983e-6276-4880-a226-0b846427a4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:38.504935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d983e-6276-4880-a226-0b846427a4b6 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:38.506189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:38.512079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0add8-f99e-4ab7-ba98-5536273a4999 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:38.512514 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0add8-f99e-4ab7-ba98-5536273a4999 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:38.512614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0add8-f99e-4ab7-ba98-5536273a4999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:38.579760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-dbb0add8-f99e-4ab7-ba98-5536273a4999 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:38.580973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-dbb0add8-f99e-4ab7-ba98-5536273a4999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:38.620710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:38.679034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:38.965748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d983e-6276-4880-a226-0b846427a4b6 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:38.967359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-007d983e-6276-4880-a226-0b846427a4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:39.070834 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d983e-6276-4880-a226-0b846427a4b6 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:39.113058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:39.150981 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b1839-ad12-4ff8-830e-2644d94881ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:39.151598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b1839-ad12-4ff8-830e-2644d94881ba service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:39.151875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b1839-ad12-4ff8-830e-2644d94881ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:39.152143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b1839-ad12-4ff8-830e-2644d94881ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:39.153805 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ec4b1839-ad12-4ff8-830e-2644d94881ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:39.267722 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b1839-ad12-4ff8-830e-2644d94881ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:39.269158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b1839-ad12-4ff8-830e-2644d94881ba service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:39.286991 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 727/1464] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:41.578143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea472c-0dfa-4f9e-93cb-b0d83ebdccc7 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:41.578143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea472c-0dfa-4f9e-93cb-b0d83ebdccc7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:41.578649 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea472c-0dfa-4f9e-93cb-b0d83ebdccc7 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:41.579897 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:41.585545 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a804425-e220-459a-b431-6fcb2f7d0155 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:41.585970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a804425-e220-459a-b431-6fcb2f7d0155 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:41.586099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a804425-e220-459a-b431-6fcb2f7d0155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:41.654822 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-2a804425-e220-459a-b431-6fcb2f7d0155 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:41.656045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2a804425-e220-459a-b431-6fcb2f7d0155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:41.674671 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:41.756080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:42.044262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea472c-0dfa-4f9e-93cb-b0d83ebdccc7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:42.045791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-38ea472c-0dfa-4f9e-93cb-b0d83ebdccc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:42.150041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea472c-0dfa-4f9e-93cb-b0d83ebdccc7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:42.220933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:42.229512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7d8a4-4c50-4c34-922c-d1a75887d571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:42.230207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7d8a4-4c50-4c34-922c-d1a75887d571 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:42.230508 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7d8a4-4c50-4c34-922c-d1a75887d571 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:42.230779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7d8a4-4c50-4c34-922c-d1a75887d571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:42.232424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3dc7d8a4-4c50-4c34-922c-d1a75887d571 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:42.342783 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7d8a4-4c50-4c34-922c-d1a75887d571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:42.344192 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7d8a4-4c50-4c34-922c-d1a75887d571 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:42.362535 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 739/1467] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:44.541806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c8b0a-57e0-480e-b39e-e3f758fd164d None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:44.542368 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c8b0a-57e0-480e-b39e-e3f758fd164d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:44.542368 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c8b0a-57e0-480e-b39e-e3f758fd164d None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:44.543726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:00:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:44.549112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c55690-3894-4057-b4c6-fdff168c5629 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:44.549554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c55690-3894-4057-b4c6-fdff168c5629 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:44.549667 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c55690-3894-4057-b4c6-fdff168c5629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:44.617018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a2c55690-3894-4057-b4c6-fdff168c5629 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:44.618859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a2c55690-3894-4057-b4c6-fdff168c5629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:44.666322 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:44.714028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:00:44.996504 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c8b0a-57e0-480e-b39e-e3f758fd164d None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:44.998039 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-c19c8b0a-57e0-480e-b39e-e3f758fd164d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:45.100324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c8b0a-57e0-480e-b39e-e3f758fd164d None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:45.269953 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:45.296706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bb54e-8fc1-4ff1-95ae-f6127aa4f6ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:45.297310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bb54e-8fc1-4ff1-95ae-f6127aa4f6ea service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:45.297586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bb54e-8fc1-4ff1-95ae-f6127aa4f6ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:45.297856 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bb54e-8fc1-4ff1-95ae-f6127aa4f6ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:45.299498 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-992bb54e-8fc1-4ff1-95ae-f6127aa4f6ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:45.408326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992bb54e-8fc1-4ff1-95ae-f6127aa4f6ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:45.409745 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992bb54e-8fc1-4ff1-95ae-f6127aa4f6ea service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:45.427483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 730/1470] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:00:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:00:54.711898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=22 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:55.425205 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=66 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:05.422734 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a33a24-398f-4ba2-b1c0-795e9d5235a7 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:05.423306 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a33a24-398f-4ba2-b1c0-795e9d5235a7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:05.423306 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a33a24-398f-4ba2-b1c0-795e9d5235a7 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:05.424575 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 741/1471] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 12:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:05.429785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed86fb-0d71-40dc-b641-6893825914e0 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:05.430241 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed86fb-0d71-40dc-b641-6893825914e0 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:05.430346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed86fb-0d71-40dc-b641-6893825914e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:05.496602 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-2aed86fb-0d71-40dc-b641-6893825914e0 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:05.497841 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2aed86fb-0d71-40dc-b641-6893825914e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:05.595344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 731/1472] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:01:05.896798 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387905aa-b9bd-4b36-8b2f-2e9c3d41a546 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:05.897078 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387905aa-b9bd-4b36-8b2f-2e9c3d41a546 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:05.897348 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387905aa-b9bd-4b36-8b2f-2e9c3d41a546 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:05.898814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 742/1473] 10.222.0.62 () {54 vars in 966 bytes} [Fri Sep 9 12:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:05.904460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1916d-c9f4-445d-8172-caf70ef6d4cf None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:05.904741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1916d-c9f4-445d-8172-caf70ef6d4cf None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:05.905014 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1916d-c9f4-445d-8172-caf70ef6d4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:05.981739 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-44a1916d-c9f4-445d-8172-caf70ef6d4cf None None] MFA Rules not processed for user `2d477a49a1724b7e8a4699a5ebd04bde`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:05.982966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-44a1916d-c9f4-445d-8172-caf70ef6d4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:06.003454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:06.059767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 732/1474] 10.222.0.62 () {58 vars in 1063 bytes} [Fri Sep 9 12:01:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:01:06.067473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387905aa-b9bd-4b36-8b2f-2e9c3d41a546 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:06.069012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-387905aa-b9bd-4b36-8b2f-2e9c3d41a546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:06.172115 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387905aa-b9bd-4b36-8b2f-2e9c3d41a546 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:06.250936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a96b82-95a2-478b-91fd-18dc8b30a6e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d477a49a1724b7e8a4699a5ebd04bde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:06.251544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a96b82-95a2-478b-91fd-18dc8b30a6e1 service glance] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:06.251823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a96b82-95a2-478b-91fd-18dc8b30a6e1 service glance] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:06.252093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a96b82-95a2-478b-91fd-18dc8b30a6e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:06.253693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-17a96b82-95a2-478b-91fd-18dc8b30a6e1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:06.308358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:06.363426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a96b82-95a2-478b-91fd-18dc8b30a6e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:06.364839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a96b82-95a2-478b-91fd-18dc8b30a6e1 service glance] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:06.383286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 743/1475] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:01:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:01:08.646732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527614e-fd9c-42ab-bfdc-e3f2fd6f97b2 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:08.647317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527614e-fd9c-42ab-bfdc-e3f2fd6f97b2 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:08.647317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527614e-fd9c-42ab-bfdc-e3f2fd6f97b2 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:08.648629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 733/1476] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 12:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:08.654560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3fec2-a091-4bb1-872f-63a493c59fb4 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:08.654986 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3fec2-a091-4bb1-872f-63a493c59fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:08.655226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3fec2-a091-4bb1-872f-63a493c59fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:08.720541 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-61c3fec2-a091-4bb1-872f-63a493c59fb4 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:08.721754 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-61c3fec2-a091-4bb1-872f-63a493c59fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:08.816937 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 744/1477] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:01:09.087290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527614e-fd9c-42ab-bfdc-e3f2fd6f97b2 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:09.088834 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f527614e-fd9c-42ab-bfdc-e3f2fd6f97b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:09.213414 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527614e-fd9c-42ab-bfdc-e3f2fd6f97b2 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:09.267776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:09.311831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca6641-cac8-4ffe-a4aa-508200c9ddc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:09.312520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca6641-cac8-4ffe-a4aa-508200c9ddc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:09.312799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca6641-cac8-4ffe-a4aa-508200c9ddc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:09.313067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca6641-cac8-4ffe-a4aa-508200c9ddc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:09.313701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca6641-cac8-4ffe-a4aa-508200c9ddc2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:09.315475 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: WARNING py.warnings [None req-c0ca6641-cac8-4ffe-a4aa-508200c9ddc2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:01:09.315475 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: warnings.warn(msg) Sep 09 12:01:09.315475 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: Sep 09 12:01:09.316610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca6641-cac8-4ffe-a4aa-508200c9ddc2 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:09.335998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 734/1478] 10.222.0.62 () {56 vars in 1108 bytes} [Fri Sep 9 12:01:09 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:01:11.532286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bcf0d-94c0-44b2-b7b2-597004ae60c3 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:11.532816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bcf0d-94c0-44b2-b7b2-597004ae60c3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:11.532816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bcf0d-94c0-44b2-b7b2-597004ae60c3 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:11.534171 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 745/1479] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 12:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:11.539367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2aa81-a1de-45cd-a75c-904b1cf3f812 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:11.539801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2aa81-a1de-45cd-a75c-904b1cf3f812 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:11.539937 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2aa81-a1de-45cd-a75c-904b1cf3f812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:11.607585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-e8d2aa81-a1de-45cd-a75c-904b1cf3f812 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:11.608841 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e8d2aa81-a1de-45cd-a75c-904b1cf3f812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:11.708388 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 735/1480] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:01:12.005932 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bcf0d-94c0-44b2-b7b2-597004ae60c3 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:12.007486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-473bcf0d-94c0-44b2-b7b2-597004ae60c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:12.017727 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:12.154259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bcf0d-94c0-44b2-b7b2-597004ae60c3 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:12.236241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:12.270022 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b6de8-7d2f-478b-9140-91e03905fd24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:12.270661 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b6de8-7d2f-478b-9140-91e03905fd24 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:12.270938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b6de8-7d2f-478b-9140-91e03905fd24 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:12.271210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b6de8-7d2f-478b-9140-91e03905fd24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:12.272811 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0b2b6de8-7d2f-478b-9140-91e03905fd24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:12.362941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b6de8-7d2f-478b-9140-91e03905fd24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:12.364518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b6de8-7d2f-478b-9140-91e03905fd24 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:12.382825 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 746/1481] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:01:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:01:14.723436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028b898-118f-440b-8dd9-2a7ac2088570 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:14.723994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028b898-118f-440b-8dd9-2a7ac2088570 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:14.723994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028b898-118f-440b-8dd9-2a7ac2088570 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:14.725355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 736/1482] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 12:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:14.730805 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ddf91-cdff-418f-a201-2873999cedd6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:14.731227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ddf91-cdff-418f-a201-2873999cedd6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:14.731333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ddf91-cdff-418f-a201-2873999cedd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:14.797798 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c67ddf91-cdff-418f-a201-2873999cedd6 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:14.799047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c67ddf91-cdff-418f-a201-2873999cedd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:14.853283 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:14.905417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 747/1483] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:01:15.191011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028b898-118f-440b-8dd9-2a7ac2088570 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:15.192665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-1028b898-118f-440b-8dd9-2a7ac2088570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:15.216877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:15.299034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028b898-118f-440b-8dd9-2a7ac2088570 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:15.372638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574cf308-1c8a-4a0b-a3cc-01a46d9ff9c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:15.373266 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cf308-1c8a-4a0b-a3cc-01a46d9ff9c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:15.373543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cf308-1c8a-4a0b-a3cc-01a46d9ff9c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:15.373812 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cf308-1c8a-4a0b-a3cc-01a46d9ff9c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:15.375464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-574cf308-1c8a-4a0b-a3cc-01a46d9ff9c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:15.449966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:15.485449 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574cf308-1c8a-4a0b-a3cc-01a46d9ff9c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:15.487204 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574cf308-1c8a-4a0b-a3cc-01a46d9ff9c9 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:15.505781 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 737/1484] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:01:17.749302 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0436c-cd50-4053-a1c3-ea16168a0fa1 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:17.749876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0436c-cd50-4053-a1c3-ea16168a0fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:17.749876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0436c-cd50-4053-a1c3-ea16168a0fa1 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:17.751253 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 748/1485] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 12:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:17.756621 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bb0cb-434c-463c-8821-0cb3afee538d None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:17.757029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bb0cb-434c-463c-8821-0cb3afee538d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:17.757171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bb0cb-434c-463c-8821-0cb3afee538d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:17.823297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-e59bb0cb-434c-463c-8821-0cb3afee538d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:17.824436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e59bb0cb-434c-463c-8821-0cb3afee538d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:17.921362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 738/1486] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:01:18.214831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0436c-cd50-4053-a1c3-ea16168a0fa1 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:18.216366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fcc0436c-cd50-4053-a1c3-ea16168a0fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:18.319967 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0436c-cd50-4053-a1c3-ea16168a0fa1 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:18.392060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:18.393872 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375ad1d-21be-4a6c-afc8-ab9d79956c61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:18.394625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375ad1d-21be-4a6c-afc8-ab9d79956c61 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:18.395075 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375ad1d-21be-4a6c-afc8-ab9d79956c61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:18.395395 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375ad1d-21be-4a6c-afc8-ab9d79956c61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:18.397434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3375ad1d-21be-4a6c-afc8-ab9d79956c61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:18.508004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375ad1d-21be-4a6c-afc8-ab9d79956c61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:18.509397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375ad1d-21be-4a6c-afc8-ab9d79956c61 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:18.527288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 749/1487] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:01:20.838650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4a82d-03e5-46c1-b09f-7f0e9890ef02 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:20.839233 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4a82d-03e5-46c1-b09f-7f0e9890ef02 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:20.839233 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4a82d-03e5-46c1-b09f-7f0e9890ef02 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:20.840509 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 739/1488] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 12:01:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:20.847226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9121dff-10aa-463a-88d2-45738c922607 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:20.847629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9121dff-10aa-463a-88d2-45738c922607 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:20.847776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9121dff-10aa-463a-88d2-45738c922607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:20.914908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d9121dff-10aa-463a-88d2-45738c922607 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:20.916146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d9121dff-10aa-463a-88d2-45738c922607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:20.968799 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:21.023636 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 750/1489] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:01:21.317008 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4a82d-03e5-46c1-b09f-7f0e9890ef02 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:21.318592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-afb4a82d-03e5-46c1-b09f-7f0e9890ef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:21.356477 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:21.425130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4a82d-03e5-46c1-b09f-7f0e9890ef02 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:21.502850 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143401d-1a63-4d5b-9411-4356ec7bc899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:21.503377 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143401d-1a63-4d5b-9411-4356ec7bc899 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:21.503655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143401d-1a63-4d5b-9411-4356ec7bc899 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:21.503926 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143401d-1a63-4d5b-9411-4356ec7bc899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:21.505551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7143401d-1a63-4d5b-9411-4356ec7bc899 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:21.585672 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:21.617626 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143401d-1a63-4d5b-9411-4356ec7bc899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:21.619714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143401d-1a63-4d5b-9411-4356ec7bc899 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:21.637485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 740/1490] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:01:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:01:24.019778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145aae3-3ceb-4b66-98b6-57be36ea63a7 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:24.020320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145aae3-3ceb-4b66-98b6-57be36ea63a7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:24.020320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145aae3-3ceb-4b66-98b6-57be36ea63a7 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:24.021751 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 751/1491] 10.222.0.62 () {54 vars in 927 bytes} [Fri Sep 9 12:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:24.028833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2477a5-9e76-4315-8b93-9c00ef7d9b77 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:24.029333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2477a5-9e76-4315-8b93-9c00ef7d9b77 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:24.029600 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2477a5-9e76-4315-8b93-9c00ef7d9b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:24.095673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-9e2477a5-9e76-4315-8b93-9c00ef7d9b77 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:24.096884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-9e2477a5-9e76-4315-8b93-9c00ef7d9b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:24.192422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 741/1492] 10.222.0.62 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:01:24.485518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145aae3-3ceb-4b66-98b6-57be36ea63a7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:24.487087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8145aae3-3ceb-4b66-98b6-57be36ea63a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:24.592523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145aae3-3ceb-4b66-98b6-57be36ea63a7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:24.668902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e1bd4-f35c-4464-9a8f-667134c9f5ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:24.669503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e1bd4-f35c-4464-9a8f-667134c9f5ab service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:24.669771 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e1bd4-f35c-4464-9a8f-667134c9f5ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:24.670041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e1bd4-f35c-4464-9a8f-667134c9f5ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:24.671636 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-818e1bd4-f35c-4464-9a8f-667134c9f5ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:24.680074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:24.782278 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e1bd4-f35c-4464-9a8f-667134c9f5ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:24.783624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e1bd4-f35c-4464-9a8f-667134c9f5ab service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:24.802820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 752/1493] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:01:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:01:34.194400 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=82 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:34.800197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=50 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:37.943145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fa2aa-4df8-4652-af14-7ee741c2cfcd None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:37.943751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fa2aa-4df8-4652-af14-7ee741c2cfcd None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:37.943751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fa2aa-4df8-4652-af14-7ee741c2cfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:38.029362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-5a4fa2aa-4df8-4652-af14-7ee741c2cfcd None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:02:38.030700 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-5a4fa2aa-4df8-4652-af14-7ee741c2cfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:38.132522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 742/1494] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:02:38.140538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356e714-f1cc-42a4-a4d8-dafa20d1adf7 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:38.141390 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356e714-f1cc-42a4-a4d8-dafa20d1adf7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:38.141668 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356e714-f1cc-42a4-a4d8-dafa20d1adf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:38.225821 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2356e714-f1cc-42a4-a4d8-dafa20d1adf7 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:02:38.226913 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2356e714-f1cc-42a4-a4d8-dafa20d1adf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:38.329051 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 753/1495] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:02:38.335309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fa2aa-4df8-4652-af14-7ee741c2cfcd None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:38.337569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-5a4fa2aa-4df8-4652-af14-7ee741c2cfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:38.352394 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fa2aa-4df8-4652-af14-7ee741c2cfcd None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:38.414497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:38.448539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cccfb5-c766-44cc-b180-94a76a13323b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:38.449143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cccfb5-c766-44cc-b180-94a76a13323b admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:38.449417 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cccfb5-c766-44cc-b180-94a76a13323b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:38.449683 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cccfb5-c766-44cc-b180-94a76a13323b admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:38.450376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cccfb5-c766-44cc-b180-94a76a13323b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:02:38.450676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cccfb5-c766-44cc-b180-94a76a13323b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:38.451688 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: WARNING py.warnings [None req-43cccfb5-c766-44cc-b180-94a76a13323b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:02:38.451688 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: warnings.warn(msg) Sep 09 12:02:38.451688 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: Sep 09 12:02:38.452569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cccfb5-c766-44cc-b180-94a76a13323b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:38.464412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 743/1496] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:02:38.473107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356e714-f1cc-42a4-a4d8-dafa20d1adf7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:38.482352 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356e714-f1cc-42a4-a4d8-dafa20d1adf7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:38.543651 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:38.578533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c016e-717e-4941-85fd-029459798310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:38.579144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c016e-717e-4941-85fd-029459798310 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:38.579421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c016e-717e-4941-85fd-029459798310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:38.579696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c016e-717e-4941-85fd-029459798310 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:38.580323 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c016e-717e-4941-85fd-029459798310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:38.582036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: WARNING py.warnings [None req-8c5c016e-717e-4941-85fd-029459798310 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:02:38.582036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: warnings.warn(msg) Sep 09 12:02:38.582036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: Sep 09 12:02:38.583245 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c016e-717e-4941-85fd-029459798310 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:38.639428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 754/1497] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:02:38 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:02:38.643586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cccfb5-c766-44cc-b180-94a76a13323b admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:38.645204 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-43cccfb5-c766-44cc-b180-94a76a13323b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:38.678859 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cccfb5-c766-44cc-b180-94a76a13323b admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:38.773290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a947e-0403-4c12-a048-495d415e7da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:38.773829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a947e-0403-4c12-a048-495d415e7da2 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:38.774147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a947e-0403-4c12-a048-495d415e7da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:38.774444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a947e-0403-4c12-a048-495d415e7da2 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:38.775129 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a947e-0403-4c12-a048-495d415e7da2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:38.776725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: WARNING py.warnings [None req-2a7a947e-0403-4c12-a048-495d415e7da2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:02:38.776725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: warnings.warn(msg) Sep 09 12:02:38.776725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: Sep 09 12:02:38.777979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a947e-0403-4c12-a048-495d415e7da2 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:38.820762 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 744/1498] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:02:38 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:02:38.824991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c016e-717e-4941-85fd-029459798310 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:38.826596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8c5c016e-717e-4941-85fd-029459798310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:38.890325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c016e-717e-4941-85fd-029459798310 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:38.974441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:38.986905 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddb3d5-1ade-40f7-80bf-5e6235604866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:38.988110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddb3d5-1ade-40f7-80bf-5e6235604866 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:38.988110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddb3d5-1ade-40f7-80bf-5e6235604866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:38.988110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddb3d5-1ade-40f7-80bf-5e6235604866 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:38.988670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ddb3d5-1ade-40f7-80bf-5e6235604866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:38.989679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: WARNING py.warnings [None req-69ddb3d5-1ade-40f7-80bf-5e6235604866 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:02:38.989679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: warnings.warn(msg) Sep 09 12:02:38.989679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: Sep 09 12:02:38.990863 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ddb3d5-1ade-40f7-80bf-5e6235604866 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:39.003154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 755/1499] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:02:39.007218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a947e-0403-4c12-a048-495d415e7da2 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:39.008743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2a7a947e-0403-4c12-a048-495d415e7da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:39.020038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a947e-0403-4c12-a048-495d415e7da2 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:39.046144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:39.115303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fb8a0-18fb-4ab8-a18f-ec0a68e5b22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:39.116175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fb8a0-18fb-4ab8-a18f-ec0a68e5b22a admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:39.116472 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fb8a0-18fb-4ab8-a18f-ec0a68e5b22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:39.116735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fb8a0-18fb-4ab8-a18f-ec0a68e5b22a admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:39.117363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fb8a0-18fb-4ab8-a18f-ec0a68e5b22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:39.118393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: WARNING py.warnings [None req-341fb8a0-18fb-4ab8-a18f-ec0a68e5b22a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:02:39.118393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: warnings.warn(msg) Sep 09 12:02:39.118393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: Sep 09 12:02:39.119266 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fb8a0-18fb-4ab8-a18f-ec0a68e5b22a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:39.131948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 745/1500] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:02:39.136713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddb3d5-1ade-40f7-80bf-5e6235604866 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:39.138283 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-69ddb3d5-1ade-40f7-80bf-5e6235604866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:39.171983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddb3d5-1ade-40f7-80bf-5e6235604866 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:39.265635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa867ffa-3857-4ad4-b8cf-cdeda0afd052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:39.266317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa867ffa-3857-4ad4-b8cf-cdeda0afd052 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:39.266591 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa867ffa-3857-4ad4-b8cf-cdeda0afd052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:39.266854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa867ffa-3857-4ad4-b8cf-cdeda0afd052 admin admin] PATH_INFO: `/v3/projects/b73a9a0f875c4481974c10b7f7e2280b/users/952b8d6e3ff74bddbde6ef9a615a3077/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:39.289299 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa867ffa-3857-4ad4-b8cf-cdeda0afd052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b73a9a0f875c4481974c10b7f7e2280b, user_id=952b8d6e3ff74bddbde6ef9a615a3077, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:39.290333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: WARNING py.warnings [None req-aa867ffa-3857-4ad4-b8cf-cdeda0afd052 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:02:39.290333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: warnings.warn(msg) Sep 09 12:02:39.290333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: Sep 09 12:02:39.292285 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa867ffa-3857-4ad4-b8cf-cdeda0afd052 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:39.316593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 756/1501] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:02:39 2022] PUT /identity/v3/projects/b73a9a0f875c4481974c10b7f7e2280b/users/952b8d6e3ff74bddbde6ef9a615a3077/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:02:39.329143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad9cdd-467b-42dc-9fc6-b4d36558e3bd None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:39.329426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad9cdd-467b-42dc-9fc6-b4d36558e3bd None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:39.329695 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad9cdd-467b-42dc-9fc6-b4d36558e3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:39.391391 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-36ad9cdd-467b-42dc-9fc6-b4d36558e3bd None None] MFA Rules not processed for user `952b8d6e3ff74bddbde6ef9a615a3077`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:02:39.393179 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-36ad9cdd-467b-42dc-9fc6-b4d36558e3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:39.472156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 746/1502] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:02:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:02:39.485322 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4c69d-bc3b-4e88-bbc6-b01d77c6fa1c None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:39.485606 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4c69d-bc3b-4e88-bbc6-b01d77c6fa1c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:39.485882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4c69d-bc3b-4e88-bbc6-b01d77c6fa1c None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:39.487283 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 757/1503] 10.222.0.62 () {54 vars in 964 bytes} [Fri Sep 9 12:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:02:39.492207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6c0f1-6916-4b1d-9ca8-ad0c6d340832 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:39.492487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6c0f1-6916-4b1d-9ca8-ad0c6d340832 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:39.492753 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6c0f1-6916-4b1d-9ca8-ad0c6d340832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:39.525230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:39.569984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-cdd6c0f1-6916-4b1d-9ca8-ad0c6d340832 None None] MFA Rules not processed for user `a25411d51a524f4b9f0ff846a0b29b6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:02:39.571159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-cdd6c0f1-6916-4b1d-9ca8-ad0c6d340832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:39.646655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 747/1504] 10.222.0.62 () {58 vars in 1061 bytes} [Fri Sep 9 12:02:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:02:39.654574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4c69d-bc3b-4e88-bbc6-b01d77c6fa1c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:39.656101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a5c4c69d-bc3b-4e88-bbc6-b01d77c6fa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:39.717138 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:39.777823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4c69d-bc3b-4e88-bbc6-b01d77c6fa1c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:39.860651 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45477a70-8780-4322-b307-84d3a62def05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:39.861263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45477a70-8780-4322-b307-84d3a62def05 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:39.861543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45477a70-8780-4322-b307-84d3a62def05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:39.861816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45477a70-8780-4322-b307-84d3a62def05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:39.863454 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-45477a70-8780-4322-b307-84d3a62def05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:39.964182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45477a70-8780-4322-b307-84d3a62def05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:39.965613 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45477a70-8780-4322-b307-84d3a62def05 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:39.969145 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:39.985129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 758/1505] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:02:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:02:40.032001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfd97d-fabe-46cf-84a0-52a825993b05 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:40.032460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfd97d-fabe-46cf-84a0-52a825993b05 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:40.032564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfd97d-fabe-46cf-84a0-52a825993b05 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:40.033809 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 748/1506] 10.222.0.62 () {52 vars in 830 bytes} [Fri Sep 9 12:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:02:40.037603 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45477a70-8780-4322-b307-84d3a62def05 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:40.038857 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-45477a70-8780-4322-b307-84d3a62def05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:40.138957 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45477a70-8780-4322-b307-84d3a62def05 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:40.234018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e22d50-aa75-4e65-a6e3-0396b05ad500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:40.234690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e22d50-aa75-4e65-a6e3-0396b05ad500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:40.234966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e22d50-aa75-4e65-a6e3-0396b05ad500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:40.235235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e22d50-aa75-4e65-a6e3-0396b05ad500 admin admin] PATH_INFO: `/v3/users/952b8d6e3ff74bddbde6ef9a615a3077` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:40.248725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e22d50-aa75-4e65-a6e3-0396b05ad500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=952b8d6e3ff74bddbde6ef9a615a3077)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:40.249752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: WARNING py.warnings [None req-e9e22d50-aa75-4e65-a6e3-0396b05ad500 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:02:40.249752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: warnings.warn(msg) Sep 09 12:02:40.249752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: Sep 09 12:02:40.250996 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e22d50-aa75-4e65-a6e3-0396b05ad500 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:40.261900 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:40.318440 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e9e22d50-aa75-4e65-a6e3-0396b05ad500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '952b8d6e3ff74bddbde6ef9a615a3077'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:02:40.318877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e9e22d50-aa75-4e65-a6e3-0396b05ad500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '952b8d6e3ff74bddbde6ef9a615a3077'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:02:40.333493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e9e22d50-aa75-4e65-a6e3-0396b05ad500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '952b8d6e3ff74bddbde6ef9a615a3077'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:02:40.343645 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e9e22d50-aa75-4e65-a6e3-0396b05ad500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '952b8d6e3ff74bddbde6ef9a615a3077'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:02:40.367845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e9e22d50-aa75-4e65-a6e3-0396b05ad500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '952b8d6e3ff74bddbde6ef9a615a3077'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:02:40.373922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 759/1507] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:02:40 2022] DELETE /identity/v3/users/952b8d6e3ff74bddbde6ef9a615a3077 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:02:40.378498 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d458a7-0450-4073-b565-4351df94f13a None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:40.378789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d458a7-0450-4073-b565-4351df94f13a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:40.379068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d458a7-0450-4073-b565-4351df94f13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:40.437767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-e7d458a7-0450-4073-b565-4351df94f13a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:02:40.438924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e7d458a7-0450-4073-b565-4351df94f13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:40.514671 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:40.527998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 749/1508] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:02:40.538668 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e22d50-aa75-4e65-a6e3-0396b05ad500 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:40.539899 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e9e22d50-aa75-4e65-a6e3-0396b05ad500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:40.643069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e22d50-aa75-4e65-a6e3-0396b05ad500 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:40.698267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:40.719968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53975b-3024-4903-9092-0959e2142db1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:40.720581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53975b-3024-4903-9092-0959e2142db1 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:40.720859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53975b-3024-4903-9092-0959e2142db1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:40.721129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53975b-3024-4903-9092-0959e2142db1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:40.722765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3e53975b-3024-4903-9092-0959e2142db1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:40.830863 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53975b-3024-4903-9092-0959e2142db1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:40.832287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53975b-3024-4903-9092-0959e2142db1 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:40.852236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 760/1509] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:02:41.174805 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d458a7-0450-4073-b565-4351df94f13a None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:41.176090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e7d458a7-0450-4073-b565-4351df94f13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:41.188106 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d458a7-0450-4073-b565-4351df94f13a None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:41.286756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855578d-0cd1-4767-921b-3d50a489e413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:41.287301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855578d-0cd1-4767-921b-3d50a489e413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:41.287579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855578d-0cd1-4767-921b-3d50a489e413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:41.287847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855578d-0cd1-4767-921b-3d50a489e413 admin admin] PATH_INFO: `/v3/projects/b73a9a0f875c4481974c10b7f7e2280b` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:41.298241 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855578d-0cd1-4767-921b-3d50a489e413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b73a9a0f875c4481974c10b7f7e2280b)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:41.299268 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: WARNING py.warnings [None req-9855578d-0cd1-4767-921b-3d50a489e413 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:02:41.299268 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: warnings.warn(msg) Sep 09 12:02:41.299268 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: Sep 09 12:02:41.300496 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855578d-0cd1-4767-921b-3d50a489e413 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:41.358829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9855578d-0cd1-4767-921b-3d50a489e413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b73a9a0f875c4481974c10b7f7e2280b', 'request_id': 'req-9855578d-0cd1-4767-921b-3d50a489e413'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:02:41.369328 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9855578d-0cd1-4767-921b-3d50a489e413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b73a9a0f875c4481974c10b7f7e2280b', 'request_id': 'req-9855578d-0cd1-4767-921b-3d50a489e413'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:02:41.376268 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9855578d-0cd1-4767-921b-3d50a489e413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b73a9a0f875c4481974c10b7f7e2280b', 'request_id': 'req-9855578d-0cd1-4767-921b-3d50a489e413'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:02:41.406412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9855578d-0cd1-4767-921b-3d50a489e413 admin admin] The token cache is being invalidate because project b73a9a0f875c4481974c10b7f7e2280b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:02:41.406706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9855578d-0cd1-4767-921b-3d50a489e413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:02:41.407140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9855578d-0cd1-4767-921b-3d50a489e413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:02:41.408573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 750/1510] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:02:41 2022] DELETE /identity/v3/projects/b73a9a0f875c4481974c10b7f7e2280b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:02:50.848667 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=62 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:51.404357 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:09.399168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf366c-b918-4652-9531-48722b0f82c4 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:09.399786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf366c-b918-4652-9531-48722b0f82c4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:09.399786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf366c-b918-4652-9531-48722b0f82c4 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:09.401104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:09.406559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d30138-f668-4b7f-a4d2-da7fe5b7d1e0 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:09.407030 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d30138-f668-4b7f-a4d2-da7fe5b7d1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:09.407139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d30138-f668-4b7f-a4d2-da7fe5b7d1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:09.487736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-62d30138-f668-4b7f-a4d2-da7fe5b7d1e0 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:09.488870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-62d30138-f668-4b7f-a4d2-da7fe5b7d1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:09.592425 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:04:09.750938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf366c-b918-4652-9531-48722b0f82c4 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:09.753063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-28cf366c-b918-4652-9531-48722b0f82c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:09.878046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf366c-b918-4652-9531-48722b0f82c4 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:09.883838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca9198-7445-443e-a20f-a25aa4fb5c72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:09.884440 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca9198-7445-443e-a20f-a25aa4fb5c72 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:09.884717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca9198-7445-443e-a20f-a25aa4fb5c72 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:09.884989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca9198-7445-443e-a20f-a25aa4fb5c72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:09.894593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca9198-7445-443e-a20f-a25aa4fb5c72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:09.896179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca9198-7445-443e-a20f-a25aa4fb5c72 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:09.905697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 762/1513] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:11.018176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babca179-046f-4b15-8ce4-366a14423725 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:11.018845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babca179-046f-4b15-8ce4-366a14423725 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:11.018845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babca179-046f-4b15-8ce4-366a14423725 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:11.020071 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:11.026569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df7cdd-65af-4639-b5d4-85a2178f56f8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:11.027611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df7cdd-65af-4639-b5d4-85a2178f56f8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:11.027719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df7cdd-65af-4639-b5d4-85a2178f56f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:11.047190 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-32df7cdd-65af-4639-b5d4-85a2178f56f8 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:11.048391 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-32df7cdd-65af-4639-b5d4-85a2178f56f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:11.056241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:04:11.210916 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babca179-046f-4b15-8ce4-366a14423725 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:11.226401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babca179-046f-4b15-8ce4-366a14423725 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:11.231914 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba52d0-9bb1-4c49-89a2-95119db9d22c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:11.232511 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba52d0-9bb1-4c49-89a2-95119db9d22c service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:11.232791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba52d0-9bb1-4c49-89a2-95119db9d22c service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:11.233063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba52d0-9bb1-4c49-89a2-95119db9d22c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:11.241541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba52d0-9bb1-4c49-89a2-95119db9d22c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:11.243437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba52d0-9bb1-4c49-89a2-95119db9d22c service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:11.253007 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 753/1516] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:11.414353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe4c3b-96ac-4ded-b1aa-94535572aef7 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:11.414637 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe4c3b-96ac-4ded-b1aa-94535572aef7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:11.414914 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe4c3b-96ac-4ded-b1aa-94535572aef7 None None] PATH_INFO: `/v3` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:11.416227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 764/1517] 10.222.0.73 () {54 vars in 925 bytes} [Fri Sep 9 12:04:11 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:04:11.422380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ee955-e2f2-4a3f-826c-ccb1bbc98364 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:11.422674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ee955-e2f2-4a3f-826c-ccb1bbc98364 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:11.422948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ee955-e2f2-4a3f-826c-ccb1bbc98364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:11.483974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-742ee955-e2f2-4a3f-826c-ccb1bbc98364 None None] MFA Rules not processed for user `c58bae3ef7784671ae247f4ad4199c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:11.485192 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-742ee955-e2f2-4a3f-826c-ccb1bbc98364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:11.502582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:11.559330 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 754/1518] 10.222.0.73 () {58 vars in 1015 bytes} [Fri Sep 9 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:04:20.410347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60c7e6-d5a0-42d8-ac1e-4de393ab7fc9 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:20.410833 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60c7e6-d5a0-42d8-ac1e-4de393ab7fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:20.410947 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60c7e6-d5a0-42d8-ac1e-4de393ab7fc9 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:20.412219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 765/1519] 10.222.0.62 () {54 vars in 961 bytes} [Fri Sep 9 12:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:20.417160 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df028d69-fb2f-4450-95a1-0c7c65012827 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:20.417557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df028d69-fb2f-4450-95a1-0c7c65012827 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:20.417710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df028d69-fb2f-4450-95a1-0c7c65012827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:20.437593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-df028d69-fb2f-4450-95a1-0c7c65012827 None None] MFA Rules not processed for user `c58bae3ef7784671ae247f4ad4199c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:20.438816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-df028d69-fb2f-4450-95a1-0c7c65012827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:20.448146 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 755/1520] 10.222.0.62 () {58 vars in 1058 bytes} [Fri Sep 9 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:04:20.456505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60c7e6-d5a0-42d8-ac1e-4de393ab7fc9 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:20.470200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60c7e6-d5a0-42d8-ac1e-4de393ab7fc9 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:20.475254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28991b58-8512-423b-acea-777c618f42d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:20.475845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28991b58-8512-423b-acea-777c618f42d0 service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:20.476119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28991b58-8512-423b-acea-777c618f42d0 service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:20.476394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28991b58-8512-423b-acea-777c618f42d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:20.484835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28991b58-8512-423b-acea-777c618f42d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:20.486405 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28991b58-8512-423b-acea-777c618f42d0 service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:20.496394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 766/1521] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:22.723216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e70c3-7640-4eb9-be5f-6994d82b7346 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:22.723811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e70c3-7640-4eb9-be5f-6994d82b7346 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:22.723811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e70c3-7640-4eb9-be5f-6994d82b7346 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:22.725109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:22.731446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355bd9b-603e-4ffd-b8c1-8bc3b377794c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:22.731881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355bd9b-603e-4ffd-b8c1-8bc3b377794c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:22.731991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355bd9b-603e-4ffd-b8c1-8bc3b377794c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:22.752348 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-3355bd9b-603e-4ffd-b8c1-8bc3b377794c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:22.753554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3355bd9b-603e-4ffd-b8c1-8bc3b377794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:22.762023 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:04:22.916677 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e70c3-7640-4eb9-be5f-6994d82b7346 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:22.931474 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e70c3-7640-4eb9-be5f-6994d82b7346 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:22.937445 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218153c-23f0-416e-87cd-161d5329e4b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:22.938028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218153c-23f0-416e-87cd-161d5329e4b7 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:22.938330 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218153c-23f0-416e-87cd-161d5329e4b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:22.938601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218153c-23f0-416e-87cd-161d5329e4b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:22.946542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218153c-23f0-416e-87cd-161d5329e4b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:22.948116 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218153c-23f0-416e-87cd-161d5329e4b7 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:22.957585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 757/1524] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:24.023329 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2b0aa-e09d-4042-8f4e-e76f627ae2f1 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:24.023789 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2b0aa-e09d-4042-8f4e-e76f627ae2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:24.023887 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2b0aa-e09d-4042-8f4e-e76f627ae2f1 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:24.025222 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:24.031835 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39a9a7-7417-4ec9-a946-0c0009fe95bc None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:24.032301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39a9a7-7417-4ec9-a946-0c0009fe95bc None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:24.032416 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39a9a7-7417-4ec9-a946-0c0009fe95bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:24.052009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-6e39a9a7-7417-4ec9-a946-0c0009fe95bc None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:24.053212 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6e39a9a7-7417-4ec9-a946-0c0009fe95bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:24.061054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:04:24.213090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2b0aa-e09d-4042-8f4e-e76f627ae2f1 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:24.228368 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2b0aa-e09d-4042-8f4e-e76f627ae2f1 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:24.234565 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632b0d0-296b-4f20-9d5e-765b72b6ad88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:24.235160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632b0d0-296b-4f20-9d5e-765b72b6ad88 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:24.235438 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632b0d0-296b-4f20-9d5e-765b72b6ad88 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:24.235708 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632b0d0-296b-4f20-9d5e-765b72b6ad88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:24.244162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632b0d0-296b-4f20-9d5e-765b72b6ad88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:24.245722 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632b0d0-296b-4f20-9d5e-765b72b6ad88 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:24.255150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 769/1527] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:28.335929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55e87b-e7c3-440d-a3e8-620962856ae0 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:28.336484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55e87b-e7c3-440d-a3e8-620962856ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:28.336484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55e87b-e7c3-440d-a3e8-620962856ae0 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:28.337819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:28.343789 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392a02e-148a-4ff7-ad08-ddb8f27f0b6e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:28.344227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392a02e-148a-4ff7-ad08-ddb8f27f0b6e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:28.344340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392a02e-148a-4ff7-ad08-ddb8f27f0b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:28.363555 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-8392a02e-148a-4ff7-ad08-ddb8f27f0b6e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:28.364753 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8392a02e-148a-4ff7-ad08-ddb8f27f0b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:28.372851 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:04:28.665878 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55e87b-e7c3-440d-a3e8-620962856ae0 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:28.667631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7f55e87b-e7c3-440d-a3e8-620962856ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:28.751673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55e87b-e7c3-440d-a3e8-620962856ae0 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:28.755316 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d0ad0-551b-4f1b-b2b9-499258449c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:28.755916 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d0ad0-551b-4f1b-b2b9-499258449c53 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:28.756190 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d0ad0-551b-4f1b-b2b9-499258449c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:28.756455 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d0ad0-551b-4f1b-b2b9-499258449c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:28.766186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d0ad0-551b-4f1b-b2b9-499258449c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:28.767623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d0ad0-551b-4f1b-b2b9-499258449c53 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:28.777209 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 760/1530] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:30.931828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f6aa2-cd78-4961-babe-f9f64797a1a6 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:30.932396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f6aa2-cd78-4961-babe-f9f64797a1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:30.932396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f6aa2-cd78-4961-babe-f9f64797a1a6 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:30.933761 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:30.939683 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3849e25-6e17-4315-ad89-3a2d781ed691 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:30.940114 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3849e25-6e17-4315-ad89-3a2d781ed691 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:30.940230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3849e25-6e17-4315-ad89-3a2d781ed691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:30.960353 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-e3849e25-6e17-4315-ad89-3a2d781ed691 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:30.961553 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e3849e25-6e17-4315-ad89-3a2d781ed691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:30.969367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:04:31.259513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f6aa2-cd78-4961-babe-f9f64797a1a6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:31.264947 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=75 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:31.271123 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f6aa2-cd78-4961-babe-f9f64797a1a6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:31.274928 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96241a-4aeb-4125-b9b4-784ba1ebf9ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:31.275561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96241a-4aeb-4125-b9b4-784ba1ebf9ad service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:31.275835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96241a-4aeb-4125-b9b4-784ba1ebf9ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:31.276102 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96241a-4aeb-4125-b9b4-784ba1ebf9ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:31.285331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96241a-4aeb-4125-b9b4-784ba1ebf9ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:31.286788 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96241a-4aeb-4125-b9b4-784ba1ebf9ad service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:31.296520 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 772/1533] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:35.000717 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf491330-5d46-4263-a3af-7834a11253ca None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:35.001300 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf491330-5d46-4263-a3af-7834a11253ca None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:35.001300 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf491330-5d46-4263-a3af-7834a11253ca None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:35.002635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:35.007927 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db0752-5371-4fb8-a286-ffd776d18169 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:35.008350 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db0752-5371-4fb8-a286-ffd776d18169 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:35.008461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db0752-5371-4fb8-a286-ffd776d18169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:35.030424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b3db0752-5371-4fb8-a286-ffd776d18169 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:35.031615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b3db0752-5371-4fb8-a286-ffd776d18169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:35.039300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:04:35.342134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf491330-5d46-4263-a3af-7834a11253ca None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:35.343849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-bf491330-5d46-4263-a3af-7834a11253ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:35.409011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:35.422565 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf491330-5d46-4263-a3af-7834a11253ca None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:35.426135 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1927e73-b802-4320-a9e6-643b7e083756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:35.426760 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1927e73-b802-4320-a9e6-643b7e083756 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:35.427039 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1927e73-b802-4320-a9e6-643b7e083756 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:35.427310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1927e73-b802-4320-a9e6-643b7e083756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:35.436853 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1927e73-b802-4320-a9e6-643b7e083756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:35.438331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1927e73-b802-4320-a9e6-643b7e083756 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:35.448011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 763/1536] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:37.614019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7c46e-2933-402a-bee1-b95d3a55b7bf None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:37.614632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7c46e-2933-402a-bee1-b95d3a55b7bf None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:37.614632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7c46e-2933-402a-bee1-b95d3a55b7bf None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:37.616010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:37.621414 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b813f9a-a14f-4da1-9463-24267fb3ffb4 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:37.621901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b813f9a-a14f-4da1-9463-24267fb3ffb4 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:37.622021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b813f9a-a14f-4da1-9463-24267fb3ffb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:37.641754 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-6b813f9a-a14f-4da1-9463-24267fb3ffb4 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:37.642989 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6b813f9a-a14f-4da1-9463-24267fb3ffb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:37.650829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:04:37.953484 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7c46e-2933-402a-bee1-b95d3a55b7bf None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:37.955180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-16b7c46e-2933-402a-bee1-b95d3a55b7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:37.968445 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7c46e-2933-402a-bee1-b95d3a55b7bf None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:37.971909 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a147a2-0d8a-4613-9fd8-7ceb6ba48a74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:37.972498 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a147a2-0d8a-4613-9fd8-7ceb6ba48a74 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:37.972772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a147a2-0d8a-4613-9fd8-7ceb6ba48a74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:37.973038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a147a2-0d8a-4613-9fd8-7ceb6ba48a74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:37.982437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a147a2-0d8a-4613-9fd8-7ceb6ba48a74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:37.983886 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a147a2-0d8a-4613-9fd8-7ceb6ba48a74 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:37.993594 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 775/1539] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:47.636767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=11 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:47.989178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=14 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:03.850007 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11c4a3-9a27-4a16-829e-608c3586e24d None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:03.851193 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11c4a3-9a27-4a16-829e-608c3586e24d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:03.851712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11c4a3-9a27-4a16-829e-608c3586e24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:03.883434 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-0b11c4a3-9a27-4a16-829e-608c3586e24d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:03.885210 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0b11c4a3-9a27-4a16-829e-608c3586e24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:03.894675 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 765/1540] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:06:03.904939 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baa036-1c9c-42ae-a47c-c59a47d90d3d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:03.906740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baa036-1c9c-42ae-a47c-c59a47d90d3d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:03.906740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baa036-1c9c-42ae-a47c-c59a47d90d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:03.915968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9fc86-2f64-47fd-9177-f476193c84e4 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:03.917234 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9fc86-2f64-47fd-9177-f476193c84e4 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:03.917741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9fc86-2f64-47fd-9177-f476193c84e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:03.943791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-05baa036-1c9c-42ae-a47c-c59a47d90d3d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:03.945215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-05baa036-1c9c-42ae-a47c-c59a47d90d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:03.949745 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-ecc9fc86-2f64-47fd-9177-f476193c84e4 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:03.951284 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ecc9fc86-2f64-47fd-9177-f476193c84e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:03.961418 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 776/1541] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:03.967091 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 766/1542] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:06:03.971298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baa036-1c9c-42ae-a47c-c59a47d90d3d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:03.975321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0ad9c-21d4-46d4-86ed-a3e5f3349f12 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:03.975905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0ad9c-21d4-46d4-86ed-a3e5f3349f12 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:03.976419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0ad9c-21d4-46d4-86ed-a3e5f3349f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:03.982063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baa036-1c9c-42ae-a47c-c59a47d90d3d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:03.995145 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47543262-1c4e-4500-965e-8325e2a2ae3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:03.995782 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47543262-1c4e-4500-965e-8325e2a2ae3e admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:03.996056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47543262-1c4e-4500-965e-8325e2a2ae3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:03.996332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47543262-1c4e-4500-965e-8325e2a2ae3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:03.997309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47543262-1c4e-4500-965e-8325e2a2ae3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:03.997767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47543262-1c4e-4500-965e-8325e2a2ae3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:03.999201 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: WARNING py.warnings [None req-47543262-1c4e-4500-965e-8325e2a2ae3e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:06:03.999201 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: warnings.warn(msg) Sep 09 12:06:03.999201 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: Sep 09 12:06:04.000448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47543262-1c4e-4500-965e-8325e2a2ae3e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.010172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f6c0ad9c-21d4-46d4-86ed-a3e5f3349f12 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:04.011846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f6c0ad9c-21d4-46d4-86ed-a3e5f3349f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:04.019486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 777/1543] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:06:04.020770 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97880ac-4fa7-4d0c-9c73-8d6dffcb76ad None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.021053 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97880ac-4fa7-4d0c-9c73-8d6dffcb76ad None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.021325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97880ac-4fa7-4d0c-9c73-8d6dffcb76ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.028667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 767/1544] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:04.033002 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0ad9c-21d4-46d4-86ed-a3e5f3349f12 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.048774 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0ad9c-21d4-46d4-86ed-a3e5f3349f12 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.059294 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387870b2-0050-4ed5-9f02-fd7c3c214a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.059546 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a97880ac-4fa7-4d0c-9c73-8d6dffcb76ad None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:04.060452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387870b2-0050-4ed5-9f02-fd7c3c214a18 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.060756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a97880ac-4fa7-4d0c-9c73-8d6dffcb76ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:04.061201 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387870b2-0050-4ed5-9f02-fd7c3c214a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.061690 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387870b2-0050-4ed5-9f02-fd7c3c214a18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.068632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387870b2-0050-4ed5-9f02-fd7c3c214a18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:04.069170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387870b2-0050-4ed5-9f02-fd7c3c214a18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.078924 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 778/1545] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:04.079840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387870b2-0050-4ed5-9f02-fd7c3c214a18 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.079999 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97880ac-4fa7-4d0c-9c73-8d6dffcb76ad None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.094446 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 768/1546] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:06:04.098667 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97880ac-4fa7-4d0c-9c73-8d6dffcb76ad None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.100546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16442ddf-93c8-4714-bd00-00748dd4a68e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.101110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16442ddf-93c8-4714-bd00-00748dd4a68e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.106339 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16442ddf-93c8-4714-bd00-00748dd4a68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.118150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e4782-825a-469b-be18-fb2a20e52c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.118150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e4782-825a-469b-be18-fb2a20e52c7d admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.118150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e4782-825a-469b-be18-fb2a20e52c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.118150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e4782-825a-469b-be18-fb2a20e52c7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.118659 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e4782-825a-469b-be18-fb2a20e52c7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.119265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e4782-825a-469b-be18-fb2a20e52c7d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.136592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-16442ddf-93c8-4714-bd00-00748dd4a68e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:04.138200 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-16442ddf-93c8-4714-bd00-00748dd4a68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:04.156942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 769/1547] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:04.158379 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16442ddf-93c8-4714-bd00-00748dd4a68e None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.175477 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16442ddf-93c8-4714-bd00-00748dd4a68e None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.184196 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dddf12d-79a9-4c1d-8442-eb5955bd9b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.185230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddf12d-79a9-4c1d-8442-eb5955bd9b83 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.185778 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddf12d-79a9-4c1d-8442-eb5955bd9b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.186393 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddf12d-79a9-4c1d-8442-eb5955bd9b83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.190432 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dddf12d-79a9-4c1d-8442-eb5955bd9b83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.198404 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: WARNING py.warnings [None req-8dddf12d-79a9-4c1d-8442-eb5955bd9b83 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:06:04.198404 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: warnings.warn(msg) Sep 09 12:06:04.198404 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: Sep 09 12:06:04.200388 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dddf12d-79a9-4c1d-8442-eb5955bd9b83 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.204490 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 779/1548] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:04.206010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db8a87-7ab0-4098-ac7d-5858971aa3a5 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.206470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db8a87-7ab0-4098-ac7d-5858971aa3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.206891 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db8a87-7ab0-4098-ac7d-5858971aa3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.247353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-62db8a87-7ab0-4098-ac7d-5858971aa3a5 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:04.248847 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-62db8a87-7ab0-4098-ac7d-5858971aa3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:04.267303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 770/1549] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:04.268626 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dddf12d-79a9-4c1d-8442-eb5955bd9b83 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.360408 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 780/1550] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:04.361716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db8a87-7ab0-4098-ac7d-5858971aa3a5 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.366953 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dddf12d-79a9-4c1d-8442-eb5955bd9b83 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.373119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ac0ca-a91a-49f1-9ace-80a962ffc2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.373743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ac0ca-a91a-49f1-9ace-80a962ffc2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.374029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ac0ca-a91a-49f1-9ace-80a962ffc2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.374335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ac0ca-a91a-49f1-9ace-80a962ffc2a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.374996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ac0ca-a91a-49f1-9ace-80a962ffc2a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:04.375733 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ac0ca-a91a-49f1-9ace-80a962ffc2a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.376634 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db8a87-7ab0-4098-ac7d-5858971aa3a5 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.377369 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ac0ca-a91a-49f1-9ace-80a962ffc2a9 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.383301 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6fc07-2740-4d62-9737-b9868acad0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.384215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6fc07-2740-4d62-9737-b9868acad0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.384836 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6fc07-2740-4d62-9737-b9868acad0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.386176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6fc07-2740-4d62-9737-b9868acad0c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.387061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6fc07-2740-4d62-9737-b9868acad0c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.388533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: WARNING py.warnings [None req-90a6fc07-2740-4d62-9737-b9868acad0c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:06:04.388533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: warnings.warn(msg) Sep 09 12:06:04.388533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: Sep 09 12:06:04.389966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6fc07-2740-4d62-9737-b9868acad0c7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.395051 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 771/1551] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:06:04.395051 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68632a13-6182-4410-82ef-398f540d685b None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.395051 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68632a13-6182-4410-82ef-398f540d685b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.395051 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68632a13-6182-4410-82ef-398f540d685b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.414179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 781/1552] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/users => generated 288 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:04.415025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6fc07-2740-4d62-9737-b9868acad0c7 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.416199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-68632a13-6182-4410-82ef-398f540d685b None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:04.418296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-68632a13-6182-4410-82ef-398f540d685b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:04.425375 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6fc07-2740-4d62-9737-b9868acad0c7 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.426203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 772/1553] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:04.427399 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ea397-02aa-4221-a5c3-f1e696e948e3 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.427681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ea397-02aa-4221-a5c3-f1e696e948e3 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.427950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ea397-02aa-4221-a5c3-f1e696e948e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.437789 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896d6ef-c5da-4c2a-b133-b82576bbbe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.438442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896d6ef-c5da-4c2a-b133-b82576bbbe6e admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.438722 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896d6ef-c5da-4c2a-b133-b82576bbbe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.438994 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896d6ef-c5da-4c2a-b133-b82576bbbe6e admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.439632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896d6ef-c5da-4c2a-b133-b82576bbbe6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.441544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896d6ef-c5da-4c2a-b133-b82576bbbe6e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.448626 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-877ea397-02aa-4221-a5c3-f1e696e948e3 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:04.449828 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-877ea397-02aa-4221-a5c3-f1e696e948e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:04.459327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 773/1554] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:04.460686 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314afa1-fbbb-4f03-b7d5-6b967ac8ecbb None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.460958 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314afa1-fbbb-4f03-b7d5-6b967ac8ecbb None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.461227 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314afa1-fbbb-4f03-b7d5-6b967ac8ecbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.469633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 782/1555] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/users => generated 294 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:04.471268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d650e57-2588-458e-8fe6-7ab79e54742a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.471610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d650e57-2588-458e-8fe6-7ab79e54742a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.471891 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d650e57-2588-458e-8fe6-7ab79e54742a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.481648 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d314afa1-fbbb-4f03-b7d5-6b967ac8ecbb None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:04.482916 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d314afa1-fbbb-4f03-b7d5-6b967ac8ecbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:04.491275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 774/1556] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:04.492205 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-8d650e57-2588-458e-8fe6-7ab79e54742a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:04.492932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5ea30-0a75-47f2-8803-ca7f13c2d26f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.493223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5ea30-0a75-47f2-8803-ca7f13c2d26f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.493499 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5ea30-0a75-47f2-8803-ca7f13c2d26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.494394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8d650e57-2588-458e-8fe6-7ab79e54742a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:04.503128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 783/1557] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:04.503964 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d650e57-2588-458e-8fe6-7ab79e54742a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.514820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d650e57-2588-458e-8fe6-7ab79e54742a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.515736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-e8b5ea30-0a75-47f2-8803-ca7f13c2d26f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:04.516952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e8b5ea30-0a75-47f2-8803-ca7f13c2d26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:04.519473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045e04b-f18c-4a4d-8852-1fd149f61ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.520161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045e04b-f18c-4a4d-8852-1fd149f61ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.520354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045e04b-f18c-4a4d-8852-1fd149f61ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.520629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045e04b-f18c-4a4d-8852-1fd149f61ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.521260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045e04b-f18c-4a4d-8852-1fd149f61ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.523194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045e04b-f18c-4a4d-8852-1fd149f61ab1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.524975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 775/1558] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:04.530334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5ea30-0a75-47f2-8803-ca7f13c2d26f None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.536351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5ea30-0a75-47f2-8803-ca7f13c2d26f None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.541135 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97e329-d3c3-4725-b801-4cab9fbcd97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.542537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97e329-d3c3-4725-b801-4cab9fbcd97f admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.542819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97e329-d3c3-4725-b801-4cab9fbcd97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.543222 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97e329-d3c3-4725-b801-4cab9fbcd97f admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.544039 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a97e329-d3c3-4725-b801-4cab9fbcd97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.545823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a97e329-d3c3-4725-b801-4cab9fbcd97f admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.554591 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 784/1559] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:04.555876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400440bd-6707-42b3-97bf-6b77c59e9e2e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.556168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400440bd-6707-42b3-97bf-6b77c59e9e2e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.556437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400440bd-6707-42b3-97bf-6b77c59e9e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.559034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 776/1560] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:04.560508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2e027-4ed1-4df0-b0f2-410f4d977bae None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.560832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2e027-4ed1-4df0-b0f2-410f4d977bae None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.561311 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2e027-4ed1-4df0-b0f2-410f4d977bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.577513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-400440bd-6707-42b3-97bf-6b77c59e9e2e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:04.578741 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-400440bd-6707-42b3-97bf-6b77c59e9e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:04.582936 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-83a2e027-4ed1-4df0-b0f2-410f4d977bae None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:04.584106 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-83a2e027-4ed1-4df0-b0f2-410f4d977bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:04.613998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:04.633955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:04.643962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 785/1561] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:04.644896 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400440bd-6707-42b3-97bf-6b77c59e9e2e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.647431 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 777/1562] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:04.648190 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2e027-4ed1-4df0-b0f2-410f4d977bae None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.657788 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400440bd-6707-42b3-97bf-6b77c59e9e2e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.658711 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2e027-4ed1-4df0-b0f2-410f4d977bae None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.662669 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1656b58-b9eb-4c30-870e-a9fcf603f329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.663279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1656b58-b9eb-4c30-870e-a9fcf603f329 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.663533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11e32c-5e6c-4aa4-8cfc-e082dfb5b510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.663658 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1656b58-b9eb-4c30-870e-a9fcf603f329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.663822 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1656b58-b9eb-4c30-870e-a9fcf603f329 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.664121 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11e32c-5e6c-4aa4-8cfc-e082dfb5b510 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.664400 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11e32c-5e6c-4aa4-8cfc-e082dfb5b510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.664506 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1656b58-b9eb-4c30-870e-a9fcf603f329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.664668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11e32c-5e6c-4aa4-8cfc-e082dfb5b510 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.665309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11e32c-5e6c-4aa4-8cfc-e082dfb5b510 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:04.665615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11e32c-5e6c-4aa4-8cfc-e082dfb5b510 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.666075 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1656b58-b9eb-4c30-870e-a9fcf603f329 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.667437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11e32c-5e6c-4aa4-8cfc-e082dfb5b510 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.676247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 786/1563] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:04.677549 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec03b6-7474-4307-9841-7c1c4e57942c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.677831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec03b6-7474-4307-9841-7c1c4e57942c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.678103 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec03b6-7474-4307-9841-7c1c4e57942c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.688531 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 778/1564] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:06:04.688531 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f46799-9ab7-4334-bfe6-29cc8d26fad7 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.688531 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f46799-9ab7-4334-bfe6-29cc8d26fad7 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.688531 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f46799-9ab7-4334-bfe6-29cc8d26fad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.701612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-29ec03b6-7474-4307-9841-7c1c4e57942c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:04.702849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-29ec03b6-7474-4307-9841-7c1c4e57942c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:04.703538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f7f46799-9ab7-4334-bfe6-29cc8d26fad7 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:04.704797 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f7f46799-9ab7-4334-bfe6-29cc8d26fad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:04.711047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 787/1565] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:04.711625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec03b6-7474-4307-9841-7c1c4e57942c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.716301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 779/1566] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:04.716301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f46799-9ab7-4334-bfe6-29cc8d26fad7 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.721083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec03b6-7474-4307-9841-7c1c4e57942c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.725722 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f46799-9ab7-4334-bfe6-29cc8d26fad7 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.725847 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec17b12-1b2c-4c67-a360-0aea289a5aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.726348 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec17b12-1b2c-4c67-a360-0aea289a5aa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.726626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec17b12-1b2c-4c67-a360-0aea289a5aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.726898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec17b12-1b2c-4c67-a360-0aea289a5aa4 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.727523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec17b12-1b2c-4c67-a360-0aea289a5aa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.729485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec17b12-1b2c-4c67-a360-0aea289a5aa4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.731747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa2b09-d82b-4a0f-be2e-47b5c031eb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.732752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa2b09-d82b-4a0f-be2e-47b5c031eb31 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.733260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa2b09-d82b-4a0f-be2e-47b5c031eb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.733768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa2b09-d82b-4a0f-be2e-47b5c031eb31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.734603 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa2b09-d82b-4a0f-be2e-47b5c031eb31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.736290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa2b09-d82b-4a0f-be2e-47b5c031eb31 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.753624 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 780/1567] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:04.753624 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa2b09-d82b-4a0f-be2e-47b5c031eb31 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.759755 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 788/1568] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/users => generated 297 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:04.762021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa2b09-d82b-4a0f-be2e-47b5c031eb31 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.764252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec17b12-1b2c-4c67-a360-0aea289a5aa4 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.766784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d3b4c-3b1e-410a-9a06-20a50f928507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.767452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d3b4c-3b1e-410a-9a06-20a50f928507 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.767802 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d3b4c-3b1e-410a-9a06-20a50f928507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.768147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d3b4c-3b1e-410a-9a06-20a50f928507 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.768862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d3b4c-3b1e-410a-9a06-20a50f928507 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:04.769238 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d3b4c-3b1e-410a-9a06-20a50f928507 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.770933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d3b4c-3b1e-410a-9a06-20a50f928507 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.773956 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec17b12-1b2c-4c67-a360-0aea289a5aa4 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.779276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99615ca-7b30-480e-ab2d-e8e8cf40e660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.779883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99615ca-7b30-480e-ab2d-e8e8cf40e660 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.780170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99615ca-7b30-480e-ab2d-e8e8cf40e660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.780441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99615ca-7b30-480e-ab2d-e8e8cf40e660 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.781090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99615ca-7b30-480e-ab2d-e8e8cf40e660 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:04.781394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99615ca-7b30-480e-ab2d-e8e8cf40e660 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.783015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99615ca-7b30-480e-ab2d-e8e8cf40e660 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.783870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 781/1569] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:06:04.784708 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d3b4c-3b1e-410a-9a06-20a50f928507 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.794766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d3b4c-3b1e-410a-9a06-20a50f928507 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.799617 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab35459-f005-4702-b355-a3f56e01da2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.800248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab35459-f005-4702-b355-a3f56e01da2a admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.800558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab35459-f005-4702-b355-a3f56e01da2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.800832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab35459-f005-4702-b355-a3f56e01da2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.801475 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab35459-f005-4702-b355-a3f56e01da2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.810828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 789/1570] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:06:04.811050 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab35459-f005-4702-b355-a3f56e01da2a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.811180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99615ca-7b30-480e-ab2d-e8e8cf40e660 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.814428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99615ca-7b30-480e-ab2d-e8e8cf40e660 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.814546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 782/1571] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:04.815346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab35459-f005-4702-b355-a3f56e01da2a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.819203 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab656c5-c34f-4cfd-80c1-07d2b8f7ed04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.819808 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab656c5-c34f-4cfd-80c1-07d2b8f7ed04 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.820091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab656c5-c34f-4cfd-80c1-07d2b8f7ed04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.820365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab656c5-c34f-4cfd-80c1-07d2b8f7ed04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.821028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab656c5-c34f-4cfd-80c1-07d2b8f7ed04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.823550 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab656c5-c34f-4cfd-80c1-07d2b8f7ed04 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.827634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab35459-f005-4702-b355-a3f56e01da2a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.832219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b513215-8555-448a-9b37-14eddd3cac9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.832816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b513215-8555-448a-9b37-14eddd3cac9d admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.833093 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b513215-8555-448a-9b37-14eddd3cac9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.833355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b513215-8555-448a-9b37-14eddd3cac9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.833994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b513215-8555-448a-9b37-14eddd3cac9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:04.834325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b513215-8555-448a-9b37-14eddd3cac9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.835908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b513215-8555-448a-9b37-14eddd3cac9d admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.849234 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 783/1572] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:06:04.850063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b513215-8555-448a-9b37-14eddd3cac9d admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.851226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 790/1573] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:04.851894 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab656c5-c34f-4cfd-80c1-07d2b8f7ed04 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.866099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b513215-8555-448a-9b37-14eddd3cac9d admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.921240 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab656c5-c34f-4cfd-80c1-07d2b8f7ed04 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.925956 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc76fb4-21b0-4f8a-8294-a02e7445adab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.926295 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e81dd-e2df-46f5-b486-9884925661a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.926663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc76fb4-21b0-4f8a-8294-a02e7445adab admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.926910 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e81dd-e2df-46f5-b486-9884925661a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.927029 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc76fb4-21b0-4f8a-8294-a02e7445adab admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.927193 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc76fb4-21b0-4f8a-8294-a02e7445adab admin admin] PATH_INFO: `/v3/projects/19d3583d753d4efab0bcd6769574420b/users/326c59915e3d4b7289472d1400730a07/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.927333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e81dd-e2df-46f5-b486-9884925661a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.927512 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e81dd-e2df-46f5-b486-9884925661a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.928127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e81dd-e2df-46f5-b486-9884925661a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:04.928459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e81dd-e2df-46f5-b486-9884925661a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.930097 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e81dd-e2df-46f5-b486-9884925661a8 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.943594 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 784/1574] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:06:04.944388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc76fb4-21b0-4f8a-8294-a02e7445adab admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d3583d753d4efab0bcd6769574420b, user_id=326c59915e3d4b7289472d1400730a07, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.944871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e81dd-e2df-46f5-b486-9884925661a8 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.947155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc76fb4-21b0-4f8a-8294-a02e7445adab admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.956799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e81dd-e2df-46f5-b486-9884925661a8 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:04.962501 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66df43-eb79-4911-bad7-4b19ff5a92ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:04.963309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66df43-eb79-4911-bad7-4b19ff5a92ae admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:04.963592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66df43-eb79-4911-bad7-4b19ff5a92ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:04.963867 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66df43-eb79-4911-bad7-4b19ff5a92ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:04.964100 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 791/1575] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:04 2022] PUT /identity/v3/projects/19d3583d753d4efab0bcd6769574420b/users/326c59915e3d4b7289472d1400730a07/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:04.968075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66df43-eb79-4911-bad7-4b19ff5a92ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:04.968219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc76fb4-21b0-4f8a-8294-a02e7445adab admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.968322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66df43-eb79-4911-bad7-4b19ff5a92ae admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:04.977165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 785/1576] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:04.977971 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66df43-eb79-4911-bad7-4b19ff5a92ae admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:04.988626 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66df43-eb79-4911-bad7-4b19ff5a92ae admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.023249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:05.035005 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc76fb4-21b0-4f8a-8294-a02e7445adab admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.039845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac92e9d-4f02-4b55-aa9e-8474e2844a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.040457 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac92e9d-4f02-4b55-aa9e-8474e2844a4a admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.040741 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac92e9d-4f02-4b55-aa9e-8474e2844a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.041017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac92e9d-4f02-4b55-aa9e-8474e2844a4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.042208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac92e9d-4f02-4b55-aa9e-8474e2844a4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.044109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac92e9d-4f02-4b55-aa9e-8474e2844a4a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.048026 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9e964-f110-4880-a2d7-4a81bf6dfe87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.048664 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9e964-f110-4880-a2d7-4a81bf6dfe87 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.048917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9e964-f110-4880-a2d7-4a81bf6dfe87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.049189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9e964-f110-4880-a2d7-4a81bf6dfe87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.049859 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9e964-f110-4880-a2d7-4a81bf6dfe87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.051944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9e964-f110-4880-a2d7-4a81bf6dfe87 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.069024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:05.071855 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 792/1577] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:05.073225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac92e9d-4f02-4b55-aa9e-8474e2844a4a admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.083610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 786/1578] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:04 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:05.084817 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9e964-f110-4880-a2d7-4a81bf6dfe87 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.140177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac92e9d-4f02-4b55-aa9e-8474e2844a4a admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.153909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9e964-f110-4880-a2d7-4a81bf6dfe87 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.159988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081047ab-d249-40a7-8bcb-0a462c7d6c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.160664 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081047ab-d249-40a7-8bcb-0a462c7d6c3b admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.160944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081047ab-d249-40a7-8bcb-0a462c7d6c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.161211 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081047ab-d249-40a7-8bcb-0a462c7d6c3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.161951 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081047ab-d249-40a7-8bcb-0a462c7d6c3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.164004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081047ab-d249-40a7-8bcb-0a462c7d6c3b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.196037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 787/1579] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:05 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:05.196958 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081047ab-d249-40a7-8bcb-0a462c7d6c3b admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.215466 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6173993-ca9a-4419-8067-bd34d13f14ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.216485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6173993-ca9a-4419-8067-bd34d13f14ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.216485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6173993-ca9a-4419-8067-bd34d13f14ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.216710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6173993-ca9a-4419-8067-bd34d13f14ca admin admin] PATH_INFO: `/v3/projects/a5a0fa0209194424a4dceeb2540037a9/users/e7f795789caf411d91ff092faa4654a0/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.231678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6173993-ca9a-4419-8067-bd34d13f14ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a0fa0209194424a4dceeb2540037a9, user_id=e7f795789caf411d91ff092faa4654a0, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.234512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6173993-ca9a-4419-8067-bd34d13f14ca admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.248572 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 793/1580] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:05 2022] PUT /identity/v3/projects/a5a0fa0209194424a4dceeb2540037a9/users/e7f795789caf411d91ff092faa4654a0/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:05.261090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6173993-ca9a-4419-8067-bd34d13f14ca admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.269288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081047ab-d249-40a7-8bcb-0a462c7d6c3b admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.326239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6173993-ca9a-4419-8067-bd34d13f14ca admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.329812 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39215282-42e3-429c-8ecd-f7085a5ba64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.330457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39215282-42e3-429c-8ecd-f7085a5ba64c admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.330739 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39215282-42e3-429c-8ecd-f7085a5ba64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.330908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ff545-553e-4e7f-bc6d-dbea9da8ffb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.331033 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39215282-42e3-429c-8ecd-f7085a5ba64c admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.331512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ff545-553e-4e7f-bc6d-dbea9da8ffb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.331622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39215282-42e3-429c-8ecd-f7085a5ba64c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.331784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ff545-553e-4e7f-bc6d-dbea9da8ffb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.332052 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ff545-553e-4e7f-bc6d-dbea9da8ffb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.332673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ff545-553e-4e7f-bc6d-dbea9da8ffb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.333538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39215282-42e3-429c-8ecd-f7085a5ba64c admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.334588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ff545-553e-4e7f-bc6d-dbea9da8ffb2 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.349570 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:05.360544 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 788/1581] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:05 2022] POST /identity/v3/users => generated 295 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:05.360544 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57491219-36e2-4df6-8f27-ade221550224 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.360544 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57491219-36e2-4df6-8f27-ade221550224 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.360544 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57491219-36e2-4df6-8f27-ade221550224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.367153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 794/1582] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:05 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:05.368038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ff545-553e-4e7f-bc6d-dbea9da8ffb2 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.384414 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-57491219-36e2-4df6-8f27-ade221550224 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:05.386310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-57491219-36e2-4df6-8f27-ade221550224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:05.398192 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:05.433859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ff545-553e-4e7f-bc6d-dbea9da8ffb2 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.438621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd7a74-9ceb-4f3b-8aef-7589842464d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.439232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd7a74-9ceb-4f3b-8aef-7589842464d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.439511 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd7a74-9ceb-4f3b-8aef-7589842464d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.439784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd7a74-9ceb-4f3b-8aef-7589842464d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.440428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd7a74-9ceb-4f3b-8aef-7589842464d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.442038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd7a74-9ceb-4f3b-8aef-7589842464d2 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.448782 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 789/1583] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:05.449702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57491219-36e2-4df6-8f27-ade221550224 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.456079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 795/1584] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:05.456079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd7a74-9ceb-4f3b-8aef-7589842464d2 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.462195 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57491219-36e2-4df6-8f27-ade221550224 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.463759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd7a74-9ceb-4f3b-8aef-7589842464d2 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.466934 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801553b4-d729-4363-ba69-9792e3387b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.467539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801553b4-d729-4363-ba69-9792e3387b3e admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.467819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801553b4-d729-4363-ba69-9792e3387b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.468090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801553b4-d729-4363-ba69-9792e3387b3e admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.468304 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ac61b-dea3-4116-9ccb-2786d17cc9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.468731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801553b4-d729-4363-ba69-9792e3387b3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.468901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ac61b-dea3-4116-9ccb-2786d17cc9b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.469180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ac61b-dea3-4116-9ccb-2786d17cc9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.469454 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ac61b-dea3-4116-9ccb-2786d17cc9b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.470086 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ac61b-dea3-4116-9ccb-2786d17cc9b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.470692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801553b4-d729-4363-ba69-9792e3387b3e admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.472006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ac61b-dea3-4116-9ccb-2786d17cc9b9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.494008 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 790/1585] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:05 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:05.495489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801553b4-d729-4363-ba69-9792e3387b3e admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.498485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 796/1586] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:05 2022] POST /identity/v3/users => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:05.498485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b583c-13bf-4429-b60f-ac7e405b711d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.498883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b583c-13bf-4429-b60f-ac7e405b711d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.499161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b583c-13bf-4429-b60f-ac7e405b711d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.508412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801553b4-d729-4363-ba69-9792e3387b3e admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.513863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c07a6-e434-4c9c-a02c-e4eb7f594233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.514507 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c07a6-e434-4c9c-a02c-e4eb7f594233 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.514789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c07a6-e434-4c9c-a02c-e4eb7f594233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.515065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c07a6-e434-4c9c-a02c-e4eb7f594233 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.515699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c07a6-e434-4c9c-a02c-e4eb7f594233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.517662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c07a6-e434-4c9c-a02c-e4eb7f594233 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.526803 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ac7b583c-13bf-4429-b60f-ac7e405b711d None None] MFA Rules not processed for user `e7f795789caf411d91ff092faa4654a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:05.527998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ac7b583c-13bf-4429-b60f-ac7e405b711d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:05.541984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 791/1587] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:05 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:05.541984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c07a6-e434-4c9c-a02c-e4eb7f594233 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.549622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c07a6-e434-4c9c-a02c-e4eb7f594233 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.554250 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02876ab-071f-4109-92c4-ee015a015f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.554834 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02876ab-071f-4109-92c4-ee015a015f78 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.555201 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02876ab-071f-4109-92c4-ee015a015f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.555473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02876ab-071f-4109-92c4-ee015a015f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.556107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02876ab-071f-4109-92c4-ee015a015f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.558207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02876ab-071f-4109-92c4-ee015a015f78 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.568658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 792/1588] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:05.569439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02876ab-071f-4109-92c4-ee015a015f78 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.580009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02876ab-071f-4109-92c4-ee015a015f78 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.584583 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306c14b-609f-4975-a8f7-0d437db1a996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.585159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306c14b-609f-4975-a8f7-0d437db1a996 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.585495 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306c14b-609f-4975-a8f7-0d437db1a996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.585734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306c14b-609f-4975-a8f7-0d437db1a996 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.586411 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306c14b-609f-4975-a8f7-0d437db1a996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.588363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306c14b-609f-4975-a8f7-0d437db1a996 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.596599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 797/1589] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:05.597418 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b583c-13bf-4429-b60f-ac7e405b711d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.611276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b583c-13bf-4429-b60f-ac7e405b711d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.614473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 793/1590] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:05 2022] POST /identity/v3/users => generated 292 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:05.615294 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306c14b-609f-4975-a8f7-0d437db1a996 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.616131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaec09f-cffc-4c7a-9af6-e5c9f0554202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.616887 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaec09f-cffc-4c7a-9af6-e5c9f0554202 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.617117 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaec09f-cffc-4c7a-9af6-e5c9f0554202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.617664 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaec09f-cffc-4c7a-9af6-e5c9f0554202 admin admin] PATH_INFO: `/v3/projects/c9860574ff3947e3bc61a76e6883107a/users/9c0f1bfeb1464f87a7f3ea4add940861/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.634450 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaec09f-cffc-4c7a-9af6-e5c9f0554202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9860574ff3947e3bc61a76e6883107a, user_id=9c0f1bfeb1464f87a7f3ea4add940861, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.637897 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaec09f-cffc-4c7a-9af6-e5c9f0554202 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.653000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 798/1591] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:05 2022] PUT /identity/v3/projects/c9860574ff3947e3bc61a76e6883107a/users/9c0f1bfeb1464f87a7f3ea4add940861/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:05.655323 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaec09f-cffc-4c7a-9af6-e5c9f0554202 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.675269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306c14b-609f-4975-a8f7-0d437db1a996 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.700522 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:05.722826 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaec09f-cffc-4c7a-9af6-e5c9f0554202 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.725905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234184a-09f8-4109-a445-0adea3508dde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.726543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234184a-09f8-4109-a445-0adea3508dde service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.726819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234184a-09f8-4109-a445-0adea3508dde service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.727086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234184a-09f8-4109-a445-0adea3508dde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.727531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88569080-794f-4487-b45d-37f1f1406598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.728141 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88569080-794f-4487-b45d-37f1f1406598 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.728422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88569080-794f-4487-b45d-37f1f1406598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.728694 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88569080-794f-4487-b45d-37f1f1406598 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.729943 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88569080-794f-4487-b45d-37f1f1406598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.731572 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88569080-794f-4487-b45d-37f1f1406598 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.736971 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234184a-09f8-4109-a445-0adea3508dde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.738467 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234184a-09f8-4109-a445-0adea3508dde service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.741737 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 799/1592] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:05.742372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88569080-794f-4487-b45d-37f1f1406598 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.749401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 794/1593] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:05.750248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234184a-09f8-4109-a445-0adea3508dde service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.752912 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88569080-794f-4487-b45d-37f1f1406598 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.759356 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c7de2-58ab-43aa-908e-8f094cae3dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.759848 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234184a-09f8-4109-a445-0adea3508dde service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.760017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c7de2-58ab-43aa-908e-8f094cae3dea admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.761073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c7de2-58ab-43aa-908e-8f094cae3dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.761371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c7de2-58ab-43aa-908e-8f094cae3dea admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.762810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c7de2-58ab-43aa-908e-8f094cae3dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.764547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c7de2-58ab-43aa-908e-8f094cae3dea admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.766850 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a4bf7-5cb1-4236-9f9e-a211bb994173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.767539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a4bf7-5cb1-4236-9f9e-a211bb994173 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.767865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a4bf7-5cb1-4236-9f9e-a211bb994173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.768375 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a4bf7-5cb1-4236-9f9e-a211bb994173 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.768910 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a4bf7-5cb1-4236-9f9e-a211bb994173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.770753 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a4bf7-5cb1-4236-9f9e-a211bb994173 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.777085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 800/1594] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:05.778290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c7de2-58ab-43aa-908e-8f094cae3dea admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.786171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 795/1595] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:05.787110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a4bf7-5cb1-4236-9f9e-a211bb994173 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.792954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:05.795587 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c7de2-58ab-43aa-908e-8f094cae3dea admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.800588 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a4bf7-5cb1-4236-9f9e-a211bb994173 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.801254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0479f-2652-4618-9147-359e209bd567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.801864 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0479f-2652-4618-9147-359e209bd567 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.802236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0479f-2652-4618-9147-359e209bd567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.802513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0479f-2652-4618-9147-359e209bd567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.803159 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0479f-2652-4618-9147-359e209bd567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.804903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0479f-2652-4618-9147-359e209bd567 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.806526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd826a-8ba1-4b06-9444-d35c426e0f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.807959 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd826a-8ba1-4b06-9444-d35c426e0f3b admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.808314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd826a-8ba1-4b06-9444-d35c426e0f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.808800 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd826a-8ba1-4b06-9444-d35c426e0f3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.809512 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd826a-8ba1-4b06-9444-d35c426e0f3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.811657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd826a-8ba1-4b06-9444-d35c426e0f3b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.820334 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 801/1596] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:05.820334 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0479f-2652-4618-9147-359e209bd567 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.831730 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0479f-2652-4618-9147-359e209bd567 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.837245 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467cb17-36e8-4fde-940f-a18697dd942e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.838140 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467cb17-36e8-4fde-940f-a18697dd942e admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.838448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467cb17-36e8-4fde-940f-a18697dd942e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.838735 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467cb17-36e8-4fde-940f-a18697dd942e admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.839379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467cb17-36e8-4fde-940f-a18697dd942e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.841232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467cb17-36e8-4fde-940f-a18697dd942e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.850254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 796/1597] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:05 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:05.852346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d74fe-5f47-4c3f-81f1-93a1c902a17f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.853089 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d74fe-5f47-4c3f-81f1-93a1c902a17f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.853566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d74fe-5f47-4c3f-81f1-93a1c902a17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.856390 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 802/1598] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:05.859376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467cb17-36e8-4fde-940f-a18697dd942e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.872729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467cb17-36e8-4fde-940f-a18697dd942e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.875513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a20d74fe-5f47-4c3f-81f1-93a1c902a17f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:05.876866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a20d74fe-5f47-4c3f-81f1-93a1c902a17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:05.943022 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d030fa5-b779-4ece-98c9-3a1fc9efccc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.943639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d030fa5-b779-4ece-98c9-3a1fc9efccc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.943969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d030fa5-b779-4ece-98c9-3a1fc9efccc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.944182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d030fa5-b779-4ece-98c9-3a1fc9efccc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.944823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d030fa5-b779-4ece-98c9-3a1fc9efccc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.946670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d030fa5-b779-4ece-98c9-3a1fc9efccc2 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.956652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 797/1599] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:05.956652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d74fe-5f47-4c3f-81f1-93a1c902a17f None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.964534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 803/1600] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:05.965471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d030fa5-b779-4ece-98c9-3a1fc9efccc2 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:05.968632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d74fe-5f47-4c3f-81f1-93a1c902a17f None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.975471 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05f944-8b38-409c-b829-b10016466f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.976544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d030fa5-b779-4ece-98c9-3a1fc9efccc2 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:05.976762 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05f944-8b38-409c-b829-b10016466f14 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.977557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05f944-8b38-409c-b829-b10016466f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.978279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05f944-8b38-409c-b829-b10016466f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.979890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05f944-8b38-409c-b829-b10016466f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.982354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51b4cb-8c12-4a88-b9f2-ea22c7019cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:05.983059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51b4cb-8c12-4a88-b9f2-ea22c7019cae admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.983405 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51b4cb-8c12-4a88-b9f2-ea22c7019cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.983671 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51b4cb-8c12-4a88-b9f2-ea22c7019cae admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.983978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05f944-8b38-409c-b829-b10016466f14 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:05.984413 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51b4cb-8c12-4a88-b9f2-ea22c7019cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:05.986504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51b4cb-8c12-4a88-b9f2-ea22c7019cae admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:06.000553 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 798/1601] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:06.000553 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05f944-8b38-409c-b829-b10016466f14 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:06.000843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 804/1602] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:06.000843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51b4cb-8c12-4a88-b9f2-ea22c7019cae admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:06.011982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51b4cb-8c12-4a88-b9f2-ea22c7019cae admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:06.013545 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05f944-8b38-409c-b829-b10016466f14 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:06.017336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfd42d-f83d-4875-8557-42eaed8b475c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:06.018022 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfd42d-f83d-4875-8557-42eaed8b475c admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:06.018418 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfd42d-f83d-4875-8557-42eaed8b475c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:06.018757 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfd42d-f83d-4875-8557-42eaed8b475c admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:06.018884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3462b-d7a6-4e85-8817-89b1835572db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:06.019599 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3462b-d7a6-4e85-8817-89b1835572db admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:06.019740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfd42d-f83d-4875-8557-42eaed8b475c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:06.019839 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3462b-d7a6-4e85-8817-89b1835572db admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:06.020073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3462b-d7a6-4e85-8817-89b1835572db admin admin] PATH_INFO: `/v3/projects/002ffc44d77748ae9c618229e21f41e8/users/9676597a1b884138bf5435f3b6cd63c1/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:06.021922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfd42d-f83d-4875-8557-42eaed8b475c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:06.040264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3462b-d7a6-4e85-8817-89b1835572db admin admin] RBAC: Authorizing `identity:create_grant(project_id=002ffc44d77748ae9c618229e21f41e8, user_id=9676597a1b884138bf5435f3b6cd63c1, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:06.042633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: WARNING py.warnings [None req-78b3462b-d7a6-4e85-8817-89b1835572db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:06:06.042633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: warnings.warn(msg) Sep 09 12:06:06.042633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: Sep 09 12:06:06.044998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3462b-d7a6-4e85-8817-89b1835572db admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:06.050644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 805/1603] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:05 2022] POST /identity/v3/users => generated 300 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:06.050644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfd42d-f83d-4875-8557-42eaed8b475c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:06.060174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfd42d-f83d-4875-8557-42eaed8b475c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:06.063617 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 799/1604] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:05 2022] PUT /identity/v3/projects/002ffc44d77748ae9c618229e21f41e8/users/9676597a1b884138bf5435f3b6cd63c1/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:06.069514 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3462b-d7a6-4e85-8817-89b1835572db admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:06.133076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd6b6a-f9f6-44d3-820f-bd205ed4606a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:06.133796 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd6b6a-f9f6-44d3-820f-bd205ed4606a admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:06.134168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd6b6a-f9f6-44d3-820f-bd205ed4606a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:06.134509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd6b6a-f9f6-44d3-820f-bd205ed4606a admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:06.135403 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd6b6a-f9f6-44d3-820f-bd205ed4606a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:06.139118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd6b6a-f9f6-44d3-820f-bd205ed4606a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:06.154096 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 806/1605] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:06.155090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd6b6a-f9f6-44d3-820f-bd205ed4606a admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:06.163625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:06.165318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3462b-d7a6-4e85-8817-89b1835572db admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:06.173711 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734b1fa-e8ff-416b-bfd7-64ea5a67c68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:06.175403 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734b1fa-e8ff-416b-bfd7-64ea5a67c68a admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:06.176688 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734b1fa-e8ff-416b-bfd7-64ea5a67c68a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:06.177131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734b1fa-e8ff-416b-bfd7-64ea5a67c68a admin admin] PATH_INFO: `/v3/projects/414b3c8f1b29468684a54865d838effe/users/9af5557669984780a427691d752922a1/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:06.211117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734b1fa-e8ff-416b-bfd7-64ea5a67c68a admin admin] RBAC: Authorizing `identity:create_grant(project_id=414b3c8f1b29468684a54865d838effe, user_id=9af5557669984780a427691d752922a1, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:06.215020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734b1fa-e8ff-416b-bfd7-64ea5a67c68a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:06.241880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 800/1606] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:06 2022] PUT /identity/v3/projects/414b3c8f1b29468684a54865d838effe/users/9af5557669984780a427691d752922a1/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:06.243104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734b1fa-e8ff-416b-bfd7-64ea5a67c68a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:06.259583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd6b6a-f9f6-44d3-820f-bd205ed4606a admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:06.282640 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:06.319699 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d86fa-4a06-4645-9e7b-3721801f1376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:06.320253 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734b1fa-e8ff-416b-bfd7-64ea5a67c68a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:06.320385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d86fa-4a06-4645-9e7b-3721801f1376 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:06.320597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d86fa-4a06-4645-9e7b-3721801f1376 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:06.320878 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d86fa-4a06-4645-9e7b-3721801f1376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:06.327764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31ff0d-2628-45e7-8e5d-b54b8337d80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:06.328518 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31ff0d-2628-45e7-8e5d-b54b8337d80c admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:06.328875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31ff0d-2628-45e7-8e5d-b54b8337d80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:06.329139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31ff0d-2628-45e7-8e5d-b54b8337d80c admin admin] PATH_INFO: `/v3/projects/594c0d336a634abc80b5f3f1e8541c99/users/0dc798c340534e5ab6a5d1737ca60f07/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:06.332602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d86fa-4a06-4645-9e7b-3721801f1376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:06.334138 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d86fa-4a06-4645-9e7b-3721801f1376 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:06.345876 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31ff0d-2628-45e7-8e5d-b54b8337d80c admin admin] RBAC: Authorizing `identity:create_grant(project_id=594c0d336a634abc80b5f3f1e8541c99, user_id=0dc798c340534e5ab6a5d1737ca60f07, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:06.348440 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 807/1607] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:06.349138 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31ff0d-2628-45e7-8e5d-b54b8337d80c admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:06.349455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d86fa-4a06-4645-9e7b-3721801f1376 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:06.360852 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d86fa-4a06-4645-9e7b-3721801f1376 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:06.366174 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 801/1608] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:06 2022] PUT /identity/v3/projects/594c0d336a634abc80b5f3f1e8541c99/users/0dc798c340534e5ab6a5d1737ca60f07/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:06.367075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31ff0d-2628-45e7-8e5d-b54b8337d80c admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:06.451318 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5151a9-f263-46e5-b591-942691e6e16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:06.452494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5151a9-f263-46e5-b591-942691e6e16d admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:06.452971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5151a9-f263-46e5-b591-942691e6e16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:06.453436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5151a9-f263-46e5-b591-942691e6e16d admin admin] PATH_INFO: `/v3/projects/0475d5537c6f4116ac0905e90153bbc7/users/bfc29d15a7344754aa53d55379e2751f/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:06.456386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31ff0d-2628-45e7-8e5d-b54b8337d80c admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:06.462982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d0569-84ba-4490-86d8-2a993ad92214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:06.463965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d0569-84ba-4490-86d8-2a993ad92214 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:06.464404 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d0569-84ba-4490-86d8-2a993ad92214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:06.464686 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d0569-84ba-4490-86d8-2a993ad92214 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:06.465738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d0569-84ba-4490-86d8-2a993ad92214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:06.468010 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d0569-84ba-4490-86d8-2a993ad92214 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:06.477157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5151a9-f263-46e5-b591-942691e6e16d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0475d5537c6f4116ac0905e90153bbc7, user_id=bfc29d15a7344754aa53d55379e2751f, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:06.481531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5151a9-f263-46e5-b591-942691e6e16d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:06.486751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 802/1609] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:06.486751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145aa12-2f95-44d3-ba8a-4e1de9159063 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:06.486751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145aa12-2f95-44d3-ba8a-4e1de9159063 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:06.486751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145aa12-2f95-44d3-ba8a-4e1de9159063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:06.509248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 808/1610] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:06 2022] PUT /identity/v3/projects/0475d5537c6f4116ac0905e90153bbc7/users/bfc29d15a7344754aa53d55379e2751f/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:06.510311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5151a9-f263-46e5-b591-942691e6e16d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:06.516804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-0145aa12-2f95-44d3-ba8a-4e1de9159063 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:06.518668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0145aa12-2f95-44d3-ba8a-4e1de9159063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:06.592457 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:06.613359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 803/1611] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:06.614949 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a499d2-75ce-40e5-96a0-94390acf6f4b None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:06.615337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a499d2-75ce-40e5-96a0-94390acf6f4b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:06.615710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a499d2-75ce-40e5-96a0-94390acf6f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:06.653131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-35a499d2-75ce-40e5-96a0-94390acf6f4b None None] MFA Rules not processed for user `9af5557669984780a427691d752922a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:06.654483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-35a499d2-75ce-40e5-96a0-94390acf6f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:06.654729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5151a9-f263-46e5-b591-942691e6e16d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:06.661278 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2661913-bfc0-4633-84dc-66508dfbb46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:06.662210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2661913-bfc0-4633-84dc-66508dfbb46c admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:06.662560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2661913-bfc0-4633-84dc-66508dfbb46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:06.663012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2661913-bfc0-4633-84dc-66508dfbb46c admin admin] PATH_INFO: `/v3/projects/51e38a10fcf8400394897df5f65428b0/users/16c99b5db9794dff83ea1b67fd238655/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:06.695336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2661913-bfc0-4633-84dc-66508dfbb46c admin admin] RBAC: Authorizing `identity:create_grant(project_id=51e38a10fcf8400394897df5f65428b0, user_id=16c99b5db9794dff83ea1b67fd238655, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:06.700281 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2661913-bfc0-4633-84dc-66508dfbb46c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:06.705633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:06.731407 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 809/1612] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:06 2022] PUT /identity/v3/projects/51e38a10fcf8400394897df5f65428b0/users/16c99b5db9794dff83ea1b67fd238655/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:06.731407 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb168d8c-eebd-41e3-ab22-f284a9026aa8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:06.732356 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb168d8c-eebd-41e3-ab22-f284a9026aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:06.732743 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb168d8c-eebd-41e3-ab22-f284a9026aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:06.774354 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 804/1613] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:06.775311 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a499d2-75ce-40e5-96a0-94390acf6f4b None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:06.796635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-cb168d8c-eebd-41e3-ab22-f284a9026aa8 None None] MFA Rules not processed for user `0dc798c340534e5ab6a5d1737ca60f07`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:06.797910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cb168d8c-eebd-41e3-ab22-f284a9026aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:06.811945 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a499d2-75ce-40e5-96a0-94390acf6f4b None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:06.933697 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf57873-63fe-46d8-98c8-16a9f5704d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:06.934482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf57873-63fe-46d8-98c8-16a9f5704d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:06.934816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf57873-63fe-46d8-98c8-16a9f5704d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:06.935095 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf57873-63fe-46d8-98c8-16a9f5704d7b admin admin] PATH_INFO: `/v3/projects/7df686f26da044c890cb4436ab466b4a/users/8dff062817544027a9f27fb85dfd038d/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:06.938697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 810/1614] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:06.940279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb7195-fa3b-40c3-8c40-4507646d069f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:06.940785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb7195-fa3b-40c3-8c40-4507646d069f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:06.941219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb7195-fa3b-40c3-8c40-4507646d069f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:06.964304 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f3eb7195-fa3b-40c3-8c40-4507646d069f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:06.968651 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f3eb7195-fa3b-40c3-8c40-4507646d069f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:06.978316 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 811/1615] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:06.979681 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb7195-fa3b-40c3-8c40-4507646d069f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:06.981642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f3eb7195-fa3b-40c3-8c40-4507646d069f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:07.002936 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf57873-63fe-46d8-98c8-16a9f5704d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df686f26da044c890cb4436ab466b4a, user_id=8dff062817544027a9f27fb85dfd038d, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:07.005984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf57873-63fe-46d8-98c8-16a9f5704d7b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:07.038437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 805/1616] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:06 2022] PUT /identity/v3/projects/7df686f26da044c890cb4436ab466b4a/users/8dff062817544027a9f27fb85dfd038d/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:07.040022 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875a027-e07b-4c5a-a221-bbad756775e7 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.040356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875a027-e07b-4c5a-a221-bbad756775e7 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.040767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875a027-e07b-4c5a-a221-bbad756775e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.066325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-6875a027-e07b-4c5a-a221-bbad756775e7 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:07.067561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6875a027-e07b-4c5a-a221-bbad756775e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:07.102629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb7195-fa3b-40c3-8c40-4507646d069f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:07.160036 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 806/1617] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:07.161947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1949e00-e03c-4a53-af10-9baff208dc52 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.162466 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1949e00-e03c-4a53-af10-9baff208dc52 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.162990 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1949e00-e03c-4a53-af10-9baff208dc52 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.165680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 807/1618] 10.222.0.62 () {54 vars in 962 bytes} [Fri Sep 9 12:06:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:07.167338 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a35943-5319-4137-a324-81a8a1ab8295 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.167803 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a35943-5319-4137-a324-81a8a1ab8295 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.168240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a35943-5319-4137-a324-81a8a1ab8295 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.174420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 808/1619] 10.222.0.62 () {54 vars in 962 bytes} [Fri Sep 9 12:06:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:07.175289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a35943-5319-4137-a324-81a8a1ab8295 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:07.185895 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8f976-5f59-4246-b351-9e729742c638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:07.186198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8f976-5f59-4246-b351-9e729742c638 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.186538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8f976-5f59-4246-b351-9e729742c638 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.186985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8f976-5f59-4246-b351-9e729742c638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.207572 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8f976-5f59-4246-b351-9e729742c638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:07.212354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8f976-5f59-4246-b351-9e729742c638 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:07.235195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 812/1620] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:07.236835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8f976-5f59-4246-b351-9e729742c638 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:07.266908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8f976-5f59-4246-b351-9e729742c638 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:07.277386 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5394d-086b-4ac1-aca3-36378ff4015a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:07.278054 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5394d-086b-4ac1-aca3-36378ff4015a admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.278441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5394d-086b-4ac1-aca3-36378ff4015a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.278748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5394d-086b-4ac1-aca3-36378ff4015a admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.279633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5394d-086b-4ac1-aca3-36378ff4015a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:07.281649 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5394d-086b-4ac1-aca3-36378ff4015a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:07.290148 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:07.298813 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a35943-5319-4137-a324-81a8a1ab8295 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:07.307821 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 813/1621] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:07.309640 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c7f07-b6b5-498b-878d-f7db5a959e7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:07.309807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f8316-d9cc-497a-b344-2eda42c2fc1a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.310046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f8316-d9cc-497a-b344-2eda42c2fc1a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.311062 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c7f07-b6b5-498b-878d-f7db5a959e7f service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.311434 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c7f07-b6b5-498b-878d-f7db5a959e7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.311801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c7f07-b6b5-498b-878d-f7db5a959e7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.312021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f8316-d9cc-497a-b344-2eda42c2fc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.325159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c7f07-b6b5-498b-878d-f7db5a959e7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:07.328319 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c7f07-b6b5-498b-878d-f7db5a959e7f service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:07.352110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 809/1622] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:07.358278 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a45798-b663-475d-a449-4ccbb70704bf None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.359387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a45798-b663-475d-a449-4ccbb70704bf None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.361700 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a45798-b663-475d-a449-4ccbb70704bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.387152 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:07.397546 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a53f8316-d9cc-497a-b344-2eda42c2fc1a None None] MFA Rules not processed for user `1b30e80c421245b5a1ed5849b28ea90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:07.408360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a53f8316-d9cc-497a-b344-2eda42c2fc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:07.415664 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-01a45798-b663-475d-a449-4ccbb70704bf None None] MFA Rules not processed for user `1b30e80c421245b5a1ed5849b28ea90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:07.417074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-01a45798-b663-475d-a449-4ccbb70704bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:07.573952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 814/1623] 10.222.0.62 () {58 vars in 1059 bytes} [Fri Sep 9 12:06:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:07.576908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 810/1624] 10.222.0.62 () {58 vars in 1059 bytes} [Fri Sep 9 12:06:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:07.577507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f8316-d9cc-497a-b344-2eda42c2fc1a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:07.577756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a45798-b663-475d-a449-4ccbb70704bf None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:07.595559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f8316-d9cc-497a-b344-2eda42c2fc1a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:07.602620 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9906d-4f79-4f60-862f-1b4db693a1ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:07.604145 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9906d-4f79-4f60-862f-1b4db693a1ba service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.604629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9906d-4f79-4f60-862f-1b4db693a1ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.605196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9906d-4f79-4f60-862f-1b4db693a1ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.605526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a45798-b663-475d-a449-4ccbb70704bf None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:07.618518 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9214cf-08d3-4171-a529-6c58bf7a52ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:07.619250 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9214cf-08d3-4171-a529-6c58bf7a52ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.620074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9214cf-08d3-4171-a529-6c58bf7a52ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.620467 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9214cf-08d3-4171-a529-6c58bf7a52ed admin admin] PATH_INFO: `/v3/projects/7df686f26da044c890cb4436ab466b4a/users/8dff062817544027a9f27fb85dfd038d/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.622961 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9214cf-08d3-4171-a529-6c58bf7a52ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df686f26da044c890cb4436ab466b4a, user_id=8dff062817544027a9f27fb85dfd038d, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:07.626166 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9214cf-08d3-4171-a529-6c58bf7a52ed admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:07.626578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9906d-4f79-4f60-862f-1b4db693a1ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:07.629904 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9906d-4f79-4f60-862f-1b4db693a1ba service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:07.645656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 815/1625] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:07.647184 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9906d-4f79-4f60-862f-1b4db693a1ba service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:07.666151 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 811/1626] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:07 2022] PUT /identity/v3/projects/7df686f26da044c890cb4436ab466b4a/users/8dff062817544027a9f27fb85dfd038d/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:07.667097 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9214cf-08d3-4171-a529-6c58bf7a52ed admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:07.675529 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9906d-4f79-4f60-862f-1b4db693a1ba service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:07.804621 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9214cf-08d3-4171-a529-6c58bf7a52ed admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:07.807999 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99177-5b1d-44a4-85ca-0de17a7026e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:07.808639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99177-5b1d-44a4-85ca-0de17a7026e5 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.808916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99177-5b1d-44a4-85ca-0de17a7026e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.809182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99177-5b1d-44a4-85ca-0de17a7026e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.820492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9832d-adc3-4700-a448-87d7cd750f22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:07.821938 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9832d-adc3-4700-a448-87d7cd750f22 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.822405 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9832d-adc3-4700-a448-87d7cd750f22 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.822741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9832d-adc3-4700-a448-87d7cd750f22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.930743 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f99177-5b1d-44a4-85ca-0de17a7026e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:07.934310 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f99177-5b1d-44a4-85ca-0de17a7026e5 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:07.947306 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9832d-adc3-4700-a448-87d7cd750f22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:07.954371 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9832d-adc3-4700-a448-87d7cd750f22 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:07.958273 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 816/1627] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:07.959983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a11de-4efb-4da9-af4e-517501b4c9c9 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.962419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a11de-4efb-4da9-af4e-517501b4c9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.962707 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a11de-4efb-4da9-af4e-517501b4c9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.976814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:07.977683 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 812/1628] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:08.022496 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-309a11de-4efb-4da9-af4e-517501b4c9c9 None None] MFA Rules not processed for user `8dff062817544027a9f27fb85dfd038d`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:08.024969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-309a11de-4efb-4da9-af4e-517501b4c9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:08.173864 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 817/1629] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:07 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:08.186969 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9832d-adc3-4700-a448-87d7cd750f22 service nova] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:08.224478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:08.328164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9832d-adc3-4700-a448-87d7cd750f22 service nova] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:08.340645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d02faf-8c37-4c67-bb52-452a6a42f344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:08.342467 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d02faf-8c37-4c67-bb52-452a6a42f344 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:08.342847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d02faf-8c37-4c67-bb52-452a6a42f344 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:08.343311 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d02faf-8c37-4c67-bb52-452a6a42f344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:08.370504 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d02faf-8c37-4c67-bb52-452a6a42f344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:08.378048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d02faf-8c37-4c67-bb52-452a6a42f344 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:08.404911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 813/1630] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:08.469862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a32784-e984-4d66-9425-e08bf49278ad None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:08.470217 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a32784-e984-4d66-9425-e08bf49278ad None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:08.470494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a32784-e984-4d66-9425-e08bf49278ad None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:08.472581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 818/1631] 10.222.0.62 () {54 vars in 912 bytes} [Fri Sep 9 12:06:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:08.481655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8599d6-99af-4d9a-b305-ea5498e63a3c None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:08.481956 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8599d6-99af-4d9a-b305-ea5498e63a3c None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:08.482299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8599d6-99af-4d9a-b305-ea5498e63a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:08.499207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5008b-052a-4b1f-9a8e-ffa5a2c1f2c5 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:08.499207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5008b-052a-4b1f-9a8e-ffa5a2c1f2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:08.499540 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5008b-052a-4b1f-9a8e-ffa5a2c1f2c5 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:08.500948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 819/1632] 10.222.0.62 () {54 vars in 912 bytes} [Fri Sep 9 12:06:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:08.501676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5008b-052a-4b1f-9a8e-ffa5a2c1f2c5 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:08.555996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-ee8599d6-99af-4d9a-b305-ea5498e63a3c None None] MFA Rules not processed for user `1b30e80c421245b5a1ed5849b28ea90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:08.558214 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ee8599d6-99af-4d9a-b305-ea5498e63a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:08.597656 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 814/1633] 10.222.0.62 () {58 vars in 1009 bytes} [Fri Sep 9 12:06:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:08.599907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53880692-64de-4cf6-b07c-82873ee6e15b None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:08.600198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53880692-64de-4cf6-b07c-82873ee6e15b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:08.600473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53880692-64de-4cf6-b07c-82873ee6e15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:08.654197 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-53880692-64de-4cf6-b07c-82873ee6e15b None None] MFA Rules not processed for user `1b30e80c421245b5a1ed5849b28ea90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:08.655519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-53880692-64de-4cf6-b07c-82873ee6e15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:08.682521 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 815/1634] 10.222.0.62 () {58 vars in 1009 bytes} [Fri Sep 9 12:06:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:08.685418 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53880692-64de-4cf6-b07c-82873ee6e15b None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:08.698291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5008b-052a-4b1f-9a8e-ffa5a2c1f2c5 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:08.717851 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d5b61-40ab-4571-ac04-b82caf8cbc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:08.718629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d5b61-40ab-4571-ac04-b82caf8cbc05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:08.718916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d5b61-40ab-4571-ac04-b82caf8cbc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:08.719191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d5b61-40ab-4571-ac04-b82caf8cbc05 admin admin] PATH_INFO: `/v3/users/e7f795789caf411d91ff092faa4654a0` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:08.721425 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53880692-64de-4cf6-b07c-82873ee6e15b None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:08.722575 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d5b61-40ab-4571-ac04-b82caf8cbc05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f795789caf411d91ff092faa4654a0)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:08.725340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d5b61-40ab-4571-ac04-b82caf8cbc05 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:08.733994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eec4eb-419d-42a0-96d8-873cdc1b67e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:08.735525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eec4eb-419d-42a0-96d8-873cdc1b67e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:08.735983 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eec4eb-419d-42a0-96d8-873cdc1b67e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:08.736425 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eec4eb-419d-42a0-96d8-873cdc1b67e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:08.769385 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eec4eb-419d-42a0-96d8-873cdc1b67e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:08.774876 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eec4eb-419d-42a0-96d8-873cdc1b67e8 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:08.804371 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 816/1635] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:08.805442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eec4eb-419d-42a0-96d8-873cdc1b67e8 service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:08.838390 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eec4eb-419d-42a0-96d8-873cdc1b67e8 service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:08.848391 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51508e-28c1-40b1-a752-958d23d84c58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:08.849702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51508e-28c1-40b1-a752-958d23d84c58 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:08.850519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51508e-28c1-40b1-a752-958d23d84c58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:08.850816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51508e-28c1-40b1-a752-958d23d84c58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:08.854258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d65d5b61-40ab-4571-ac04-b82caf8cbc05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f795789caf411d91ff092faa4654a0'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:08.854258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d65d5b61-40ab-4571-ac04-b82caf8cbc05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f795789caf411d91ff092faa4654a0'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:08.855624 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-9b51508e-28c1-40b1-a752-958d23d84c58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:08.886235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d65d5b61-40ab-4571-ac04-b82caf8cbc05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f795789caf411d91ff092faa4654a0'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:08.906240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51508e-28c1-40b1-a752-958d23d84c58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:08.907912 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51508e-28c1-40b1-a752-958d23d84c58 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:08.995569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d65d5b61-40ab-4571-ac04-b82caf8cbc05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f795789caf411d91ff092faa4654a0'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:08.995569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d65d5b61-40ab-4571-ac04-b82caf8cbc05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f795789caf411d91ff092faa4654a0'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:08.996627 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 817/1636] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:09.011548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 820/1637] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:06:08 2022] DELETE /identity/v3/users/e7f795789caf411d91ff092faa4654a0 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:09.016427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ef47f-acf8-4a89-8576-f87bdeae810a None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:09.016890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ef47f-acf8-4a89-8576-f87bdeae810a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:09.017319 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ef47f-acf8-4a89-8576-f87bdeae810a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:09.052644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a55ef47f-acf8-4a89-8576-f87bdeae810a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:09.053892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a55ef47f-acf8-4a89-8576-f87bdeae810a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:09.143682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 818/1638] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:09.201706 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d5b61-40ab-4571-ac04-b82caf8cbc05 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:09.205205 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d65d5b61-40ab-4571-ac04-b82caf8cbc05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:09.234363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:09.363094 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d5b61-40ab-4571-ac04-b82caf8cbc05 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:09.374775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611eed4-be5b-475d-905b-4d7cfa1b0d1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:09.378139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611eed4-be5b-475d-905b-4d7cfa1b0d1b service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:09.378139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611eed4-be5b-475d-905b-4d7cfa1b0d1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:09.378139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611eed4-be5b-475d-905b-4d7cfa1b0d1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:09.411536 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ef47f-acf8-4a89-8576-f87bdeae810a None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:09.411880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a55ef47f-acf8-4a89-8576-f87bdeae810a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:09.417971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611eed4-be5b-475d-905b-4d7cfa1b0d1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:09.419544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611eed4-be5b-475d-905b-4d7cfa1b0d1b service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:09.443649 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ef47f-acf8-4a89-8576-f87bdeae810a None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:09.457376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5f4e2-151b-43e7-889f-fe1a699ff12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:09.460036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 821/1639] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:09.460658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5f4e2-151b-43e7-889f-fe1a699ff12f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:09.460993 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5f4e2-151b-43e7-889f-fe1a699ff12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:09.461272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5f4e2-151b-43e7-889f-fe1a699ff12f admin admin] PATH_INFO: `/v3/users/0dc798c340534e5ab6a5d1737ca60f07` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:09.468090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5f4e2-151b-43e7-889f-fe1a699ff12f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dc798c340534e5ab6a5d1737ca60f07)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:09.469194 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: WARNING py.warnings [None req-d4b5f4e2-151b-43e7-889f-fe1a699ff12f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:06:09.469194 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: warnings.warn(msg) Sep 09 12:06:09.469194 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: Sep 09 12:06:09.471068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5f4e2-151b-43e7-889f-fe1a699ff12f admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:09.623822 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d4b5f4e2-151b-43e7-889f-fe1a699ff12f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dc798c340534e5ab6a5d1737ca60f07'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:09.634575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d4b5f4e2-151b-43e7-889f-fe1a699ff12f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dc798c340534e5ab6a5d1737ca60f07'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:09.669866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d4b5f4e2-151b-43e7-889f-fe1a699ff12f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dc798c340534e5ab6a5d1737ca60f07'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:09.701374 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:09.721725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d4b5f4e2-151b-43e7-889f-fe1a699ff12f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dc798c340534e5ab6a5d1737ca60f07'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:09.724853 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d4b5f4e2-151b-43e7-889f-fe1a699ff12f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dc798c340534e5ab6a5d1737ca60f07'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:09.767333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 819/1640] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:06:09 2022] DELETE /identity/v3/users/0dc798c340534e5ab6a5d1737ca60f07 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:09.772290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f086b-424f-45ec-901a-e6977c7457a9 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:09.773437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f086b-424f-45ec-901a-e6977c7457a9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:09.773849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f086b-424f-45ec-901a-e6977c7457a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:09.814903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-455f086b-424f-45ec-901a-e6977c7457a9 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:09.816893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-455f086b-424f-45ec-901a-e6977c7457a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:09.972365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 822/1641] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:10.017133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5f4e2-151b-43e7-889f-fe1a699ff12f admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:10.019813 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d4b5f4e2-151b-43e7-889f-fe1a699ff12f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:10.168619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5f4e2-151b-43e7-889f-fe1a699ff12f admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:10.210052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c2282-a731-471f-afff-6d2dd0460056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:10.210052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c2282-a731-471f-afff-6d2dd0460056 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:10.210052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c2282-a731-471f-afff-6d2dd0460056 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:10.210052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c2282-a731-471f-afff-6d2dd0460056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:10.238699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c2282-a731-471f-afff-6d2dd0460056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:10.240176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c2282-a731-471f-afff-6d2dd0460056 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:10.264098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 820/1642] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:11.177880 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f086b-424f-45ec-901a-e6977c7457a9 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:11.179905 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-455f086b-424f-45ec-901a-e6977c7457a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:11.201104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f086b-424f-45ec-901a-e6977c7457a9 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:11.213582 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:11.214469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:11.214901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:11.215316 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] PATH_INFO: `/v3/projects/a5a0fa0209194424a4dceeb2540037a9` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:11.216653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a0fa0209194424a4dceeb2540037a9)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:11.217828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: WARNING py.warnings [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:06:11.217828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: warnings.warn(msg) Sep 09 12:06:11.217828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: Sep 09 12:06:11.221282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:11.351411 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a0fa0209194424a4dceeb2540037a9', 'request_id': 'req-2aed4820-e283-421a-9852-3592f62bb32d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:11.362146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a0fa0209194424a4dceeb2540037a9', 'request_id': 'req-2aed4820-e283-421a-9852-3592f62bb32d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:11.370539 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a0fa0209194424a4dceeb2540037a9', 'request_id': 'req-2aed4820-e283-421a-9852-3592f62bb32d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:11.393591 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] The token cache is being invalidate because project a5a0fa0209194424a4dceeb2540037a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:06:11.394368 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:11.395942 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:11.398849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 823/1643] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:06:11 2022] DELETE /identity/v3/projects/a5a0fa0209194424a4dceeb2540037a9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:11.402981 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c2282-a731-471f-afff-6d2dd0460056 service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:11.417828 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c2282-a731-471f-afff-6d2dd0460056 service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:11.507527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a2829-75da-4981-ab87-828473fd7833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:11.508411 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a2829-75da-4981-ab87-828473fd7833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:11.509025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a2829-75da-4981-ab87-828473fd7833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:11.509457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a2829-75da-4981-ab87-828473fd7833 admin admin] PATH_INFO: `/v3/users/8dff062817544027a9f27fb85dfd038d` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:11.511383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a2829-75da-4981-ab87-828473fd7833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dff062817544027a9f27fb85dfd038d)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:11.513568 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a2829-75da-4981-ab87-828473fd7833 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:11.635740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b16a2829-75da-4981-ab87-828473fd7833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dff062817544027a9f27fb85dfd038d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:11.647946 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b16a2829-75da-4981-ab87-828473fd7833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dff062817544027a9f27fb85dfd038d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:11.670338 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:11.672271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:11.678551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b16a2829-75da-4981-ab87-828473fd7833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dff062817544027a9f27fb85dfd038d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:11.689165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b16a2829-75da-4981-ab87-828473fd7833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dff062817544027a9f27fb85dfd038d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:11.689924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b16a2829-75da-4981-ab87-828473fd7833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dff062817544027a9f27fb85dfd038d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:11.695277 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed4820-e283-421a-9852-3592f62bb32d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:11.698909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 821/1644] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:06:11 2022] DELETE /identity/v3/users/8dff062817544027a9f27fb85dfd038d => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:11.709917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f61ca-f2ae-48ad-b389-a695fc6c1a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:11.710686 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f61ca-f2ae-48ad-b389-a695fc6c1a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:11.710969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f61ca-f2ae-48ad-b389-a695fc6c1a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:11.711247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f61ca-f2ae-48ad-b389-a695fc6c1a1d admin admin] PATH_INFO: `/v3/projects/594c0d336a634abc80b5f3f1e8541c99` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:11.717492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f61ca-f2ae-48ad-b389-a695fc6c1a1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=594c0d336a634abc80b5f3f1e8541c99)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:11.719833 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f61ca-f2ae-48ad-b389-a695fc6c1a1d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:11.813770 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:11.858805 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-163f61ca-f2ae-48ad-b389-a695fc6c1a1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '594c0d336a634abc80b5f3f1e8541c99', 'request_id': 'req-163f61ca-f2ae-48ad-b389-a695fc6c1a1d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:11.881709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-163f61ca-f2ae-48ad-b389-a695fc6c1a1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '594c0d336a634abc80b5f3f1e8541c99', 'request_id': 'req-163f61ca-f2ae-48ad-b389-a695fc6c1a1d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:11.892967 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-163f61ca-f2ae-48ad-b389-a695fc6c1a1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '594c0d336a634abc80b5f3f1e8541c99', 'request_id': 'req-163f61ca-f2ae-48ad-b389-a695fc6c1a1d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:11.920070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-163f61ca-f2ae-48ad-b389-a695fc6c1a1d admin admin] The token cache is being invalidate because project 594c0d336a634abc80b5f3f1e8541c99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:06:11.920423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-163f61ca-f2ae-48ad-b389-a695fc6c1a1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:11.921108 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-163f61ca-f2ae-48ad-b389-a695fc6c1a1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:11.929772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 824/1645] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:06:11 2022] DELETE /identity/v3/projects/594c0d336a634abc80b5f3f1e8541c99 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:06:11.931342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65a28d-8cd2-4d47-ab0e-c47835befd1c None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:11.932605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65a28d-8cd2-4d47-ab0e-c47835befd1c None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:11.932890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65a28d-8cd2-4d47-ab0e-c47835befd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:12.014895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-4b65a28d-8cd2-4d47-ab0e-c47835befd1c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:12.018753 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-4b65a28d-8cd2-4d47-ab0e-c47835befd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:12.064858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:12.214194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496244f-e855-4fc1-ba9d-64deb3d9ebe1 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:12.214742 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496244f-e855-4fc1-ba9d-64deb3d9ebe1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:12.214742 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496244f-e855-4fc1-ba9d-64deb3d9ebe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:12.223663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 822/1646] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:12.253668 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-6496244f-e855-4fc1-ba9d-64deb3d9ebe1 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:12.255018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6496244f-e855-4fc1-ba9d-64deb3d9ebe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:12.269506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65a28d-8cd2-4d47-ab0e-c47835befd1c None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:12.279608 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65a28d-8cd2-4d47-ab0e-c47835befd1c None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:12.280647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 825/1647] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:12.285913 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e748e-3867-4a8a-9d4e-d1154dc798a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:12.286646 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e748e-3867-4a8a-9d4e-d1154dc798a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:12.286922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e748e-3867-4a8a-9d4e-d1154dc798a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:12.287192 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e748e-3867-4a8a-9d4e-d1154dc798a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:12.287903 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e748e-3867-4a8a-9d4e-d1154dc798a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:12.288313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e748e-3867-4a8a-9d4e-d1154dc798a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:12.289977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e748e-3867-4a8a-9d4e-d1154dc798a6 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:12.313334 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496244f-e855-4fc1-ba9d-64deb3d9ebe1 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:12.319547 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 823/1648] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:06:12.330311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496244f-e855-4fc1-ba9d-64deb3d9ebe1 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:12.335670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3405507-cc9e-4393-b987-d225ec27403a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:12.335859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3405507-cc9e-4393-b987-d225ec27403a admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:12.336162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3405507-cc9e-4393-b987-d225ec27403a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:12.336381 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3405507-cc9e-4393-b987-d225ec27403a admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:12.337081 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3405507-cc9e-4393-b987-d225ec27403a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:12.339437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3405507-cc9e-4393-b987-d225ec27403a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:12.449755 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 826/1649] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:12 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:12.453324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e748e-3867-4a8a-9d4e-d1154dc798a6 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:12.514405 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e748e-3867-4a8a-9d4e-d1154dc798a6 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:12.671099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0647f58-68e6-464f-9779-a237f25c39fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:12.672159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0647f58-68e6-464f-9779-a237f25c39fd admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:12.672747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0647f58-68e6-464f-9779-a237f25c39fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:12.673246 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0647f58-68e6-464f-9779-a237f25c39fd admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:12.674456 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0647f58-68e6-464f-9779-a237f25c39fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:12.678422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0647f58-68e6-464f-9779-a237f25c39fd admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:12.743805 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3405507-cc9e-4393-b987-d225ec27403a admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:12.756893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 824/1650] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:12 2022] POST /identity/v3/users => generated 287 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:12.772662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3405507-cc9e-4393-b987-d225ec27403a admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:12.795076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b31fc-d085-4b41-8745-eef5b26d39e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:12.796260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b31fc-d085-4b41-8745-eef5b26d39e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:12.796452 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b31fc-d085-4b41-8745-eef5b26d39e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:12.796691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b31fc-d085-4b41-8745-eef5b26d39e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:12.797611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b31fc-d085-4b41-8745-eef5b26d39e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:12.799386 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b31fc-d085-4b41-8745-eef5b26d39e0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:12.820691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0647f58-68e6-464f-9779-a237f25c39fd admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:12.828243 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 827/1651] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:12.833548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0647f58-68e6-464f-9779-a237f25c39fd admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:12.847829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325e85e-cce3-4db5-a2cb-7a238fab907e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:12.848385 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325e85e-cce3-4db5-a2cb-7a238fab907e admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:12.848758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325e85e-cce3-4db5-a2cb-7a238fab907e admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:12.849124 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325e85e-cce3-4db5-a2cb-7a238fab907e admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:12.850134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325e85e-cce3-4db5-a2cb-7a238fab907e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:12.854772 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325e85e-cce3-4db5-a2cb-7a238fab907e admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:12.876344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b31fc-d085-4b41-8745-eef5b26d39e0 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:12.886685 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 825/1652] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:12.892690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b31fc-d085-4b41-8745-eef5b26d39e0 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:12.898689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21b5fe-7cd4-46ac-b024-f91737ef311e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:12.899357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21b5fe-7cd4-46ac-b024-f91737ef311e admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:12.899959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21b5fe-7cd4-46ac-b024-f91737ef311e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:12.900339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21b5fe-7cd4-46ac-b024-f91737ef311e admin admin] PATH_INFO: `/v3/projects/80ec88ab4b5246b1aa8cecc9d2373d86/users/391214020f734c8bb83e43c583c77965/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:12.943288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21b5fe-7cd4-46ac-b024-f91737ef311e admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ec88ab4b5246b1aa8cecc9d2373d86, user_id=391214020f734c8bb83e43c583c77965, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:12.950056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21b5fe-7cd4-46ac-b024-f91737ef311e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:12.979716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 828/1653] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:12 2022] PUT /identity/v3/projects/80ec88ab4b5246b1aa8cecc9d2373d86/users/391214020f734c8bb83e43c583c77965/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:12.991873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42250b-8668-46d2-adf8-f43c0d316e64 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:12.992329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42250b-8668-46d2-adf8-f43c0d316e64 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:12.992891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42250b-8668-46d2-adf8-f43c0d316e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:13.062280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-6a42250b-8668-46d2-adf8-f43c0d316e64 None None] MFA Rules not processed for user `391214020f734c8bb83e43c583c77965`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:13.073425 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6a42250b-8668-46d2-adf8-f43c0d316e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:13.246039 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 826/1654] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:13.249640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21b5fe-7cd4-46ac-b024-f91737ef311e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:13.493091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21b5fe-7cd4-46ac-b024-f91737ef311e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:13.506064 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbec52-a663-4e1f-ba3a-50d4bc195ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:13.507274 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbec52-a663-4e1f-ba3a-50d4bc195ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:13.507926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbec52-a663-4e1f-ba3a-50d4bc195ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:13.508493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbec52-a663-4e1f-ba3a-50d4bc195ed1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:13.509347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbec52-a663-4e1f-ba3a-50d4bc195ed1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:13.512918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbec52-a663-4e1f-ba3a-50d4bc195ed1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:13.575859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 829/1655] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:13 2022] POST /identity/v3/projects => generated 373 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:13.577089 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42250b-8668-46d2-adf8-f43c0d316e64 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:13.592110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42250b-8668-46d2-adf8-f43c0d316e64 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:13.602570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:13.670768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4fc988-2f6a-4f4a-9fc1-f20aa2621b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:13.671789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fc988-2f6a-4f4a-9fc1-f20aa2621b7f admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:13.672570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fc988-2f6a-4f4a-9fc1-f20aa2621b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:13.672977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fc988-2f6a-4f4a-9fc1-f20aa2621b7f admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:13.673767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4fc988-2f6a-4f4a-9fc1-f20aa2621b7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:13.675930 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4fc988-2f6a-4f4a-9fc1-f20aa2621b7f admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:13.712789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 827/1656] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:13 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:13.716326 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbec52-a663-4e1f-ba3a-50d4bc195ed1 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:13.729959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbec52-a663-4e1f-ba3a-50d4bc195ed1 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:13.736122 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d88e3-7fc2-49ba-8965-fa647c3bfcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:13.736963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d88e3-7fc2-49ba-8965-fa647c3bfcec admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:13.737374 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d88e3-7fc2-49ba-8965-fa647c3bfcec admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:13.737776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d88e3-7fc2-49ba-8965-fa647c3bfcec admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:13.738676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d88e3-7fc2-49ba-8965-fa647c3bfcec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:13.740907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d88e3-7fc2-49ba-8965-fa647c3bfcec admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:13.762608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 830/1657] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:13.766585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4fc988-2f6a-4f4a-9fc1-f20aa2621b7f admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:13.782139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4fc988-2f6a-4f4a-9fc1-f20aa2621b7f admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:13.796007 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762a503-2050-4cee-b699-d5ca1a09831b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:13.796884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762a503-2050-4cee-b699-d5ca1a09831b admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:13.797125 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762a503-2050-4cee-b699-d5ca1a09831b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:13.797405 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762a503-2050-4cee-b699-d5ca1a09831b admin admin] PATH_INFO: `/v3/projects/64f7dde881d84ef6a302a6d57ecd6cfa/users/d89cd45362b24150abfdeead616033ec/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:13.820479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762a503-2050-4cee-b699-d5ca1a09831b admin admin] RBAC: Authorizing `identity:create_grant(project_id=64f7dde881d84ef6a302a6d57ecd6cfa, user_id=d89cd45362b24150abfdeead616033ec, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:13.825618 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762a503-2050-4cee-b699-d5ca1a09831b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:13.831885 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d88e3-7fc2-49ba-8965-fa647c3bfcec admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:13.834336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-933d88e3-7fc2-49ba-8965-fa647c3bfcec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:13.861702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d88e3-7fc2-49ba-8965-fa647c3bfcec admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:13.864730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 828/1658] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:13 2022] PUT /identity/v3/projects/64f7dde881d84ef6a302a6d57ecd6cfa/users/d89cd45362b24150abfdeead616033ec/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:13.875539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762a503-2050-4cee-b699-d5ca1a09831b admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:13.930989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:14.036371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239e7e3-fdd9-458b-bcf0-c7b65e26f85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:14.038056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239e7e3-fdd9-458b-bcf0-c7b65e26f85e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:14.039554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239e7e3-fdd9-458b-bcf0-c7b65e26f85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:14.039837 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239e7e3-fdd9-458b-bcf0-c7b65e26f85e admin admin] PATH_INFO: `/v3/projects/7df686f26da044c890cb4436ab466b4a` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:14.043358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239e7e3-fdd9-458b-bcf0-c7b65e26f85e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7df686f26da044c890cb4436ab466b4a)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:14.046769 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239e7e3-fdd9-458b-bcf0-c7b65e26f85e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:14.068980 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762a503-2050-4cee-b699-d5ca1a09831b admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:14.084016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36692498-5fb4-4de4-b0bb-64897c1e86f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:14.085852 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36692498-5fb4-4de4-b0bb-64897c1e86f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:14.085941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36692498-5fb4-4de4-b0bb-64897c1e86f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:14.086301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36692498-5fb4-4de4-b0bb-64897c1e86f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:14.087899 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36692498-5fb4-4de4-b0bb-64897c1e86f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:14.090609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36692498-5fb4-4de4-b0bb-64897c1e86f5 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:14.110476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 829/1659] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:14.111415 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36692498-5fb4-4de4-b0bb-64897c1e86f5 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:14.134682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36692498-5fb4-4de4-b0bb-64897c1e86f5 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:14.136344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d239e7e3-fdd9-458b-bcf0-c7b65e26f85e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7df686f26da044c890cb4436ab466b4a', 'request_id': 'req-d239e7e3-fdd9-458b-bcf0-c7b65e26f85e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:14.151333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d239e7e3-fdd9-458b-bcf0-c7b65e26f85e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7df686f26da044c890cb4436ab466b4a', 'request_id': 'req-d239e7e3-fdd9-458b-bcf0-c7b65e26f85e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:14.164407 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d239e7e3-fdd9-458b-bcf0-c7b65e26f85e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7df686f26da044c890cb4436ab466b4a', 'request_id': 'req-d239e7e3-fdd9-458b-bcf0-c7b65e26f85e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:14.197435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d239e7e3-fdd9-458b-bcf0-c7b65e26f85e admin admin] The token cache is being invalidate because project 7df686f26da044c890cb4436ab466b4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:06:14.198846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d239e7e3-fdd9-458b-bcf0-c7b65e26f85e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:14.198846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d239e7e3-fdd9-458b-bcf0-c7b65e26f85e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:14.205941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 831/1660] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:06:13 2022] DELETE /identity/v3/projects/7df686f26da044c890cb4436ab466b4a => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:14.208063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b557fe-a698-4f00-8a6b-c999e09eb5ba None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:14.208063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b557fe-a698-4f00-8a6b-c999e09eb5ba None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:14.208063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b557fe-a698-4f00-8a6b-c999e09eb5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:14.249927 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-22b557fe-a698-4f00-8a6b-c999e09eb5ba None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:14.252697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-22b557fe-a698-4f00-8a6b-c999e09eb5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:14.265430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646632d-6a5e-4881-acdb-4fe840c80f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:14.266810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646632d-6a5e-4881-acdb-4fe840c80f77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:14.267221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646632d-6a5e-4881-acdb-4fe840c80f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:14.267625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646632d-6a5e-4881-acdb-4fe840c80f77 admin admin] PATH_INFO: `/v3/projects/64f7dde881d84ef6a302a6d57ecd6cfa/users/d89cd45362b24150abfdeead616033ec/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:14.273167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646632d-6a5e-4881-acdb-4fe840c80f77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64f7dde881d84ef6a302a6d57ecd6cfa, user_id=d89cd45362b24150abfdeead616033ec, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:14.276877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646632d-6a5e-4881-acdb-4fe840c80f77 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:14.289406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 832/1661] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:14.300795 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e260908-1c20-45b7-bd98-0effa7b6e5e6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:14.302048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e260908-1c20-45b7-bd98-0effa7b6e5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:14.304225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e260908-1c20-45b7-bd98-0effa7b6e5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:14.333849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 830/1662] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:14 2022] PUT /identity/v3/projects/64f7dde881d84ef6a302a6d57ecd6cfa/users/d89cd45362b24150abfdeead616033ec/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:14.335562 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f7dec-248d-4343-976d-cdb8718e1942 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:14.335800 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f7dec-248d-4343-976d-cdb8718e1942 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:14.336074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f7dec-248d-4343-976d-cdb8718e1942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:14.366354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2e260908-1c20-45b7-bd98-0effa7b6e5e6 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:14.368616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2e260908-1c20-45b7-bd98-0effa7b6e5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:14.408978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:14.446382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-211f7dec-248d-4343-976d-cdb8718e1942 None None] MFA Rules not processed for user `d89cd45362b24150abfdeead616033ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:14.447207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-211f7dec-248d-4343-976d-cdb8718e1942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:14.552802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 833/1663] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:14.558393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e260908-1c20-45b7-bd98-0effa7b6e5e6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:14.580887 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e260908-1c20-45b7-bd98-0effa7b6e5e6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:14.612243 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173824b8-ccde-4e38-bf97-87e282001b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:14.614010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173824b8-ccde-4e38-bf97-87e282001b06 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:14.614631 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173824b8-ccde-4e38-bf97-87e282001b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:14.615359 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173824b8-ccde-4e38-bf97-87e282001b06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:14.616527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173824b8-ccde-4e38-bf97-87e282001b06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:14.624453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173824b8-ccde-4e38-bf97-87e282001b06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:14.624453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173824b8-ccde-4e38-bf97-87e282001b06 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:14.639426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 831/1664] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:14.653984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f7dec-248d-4343-976d-cdb8718e1942 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:14.655796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-211f7dec-248d-4343-976d-cdb8718e1942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:14.678078 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 834/1665] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:06:14.679204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173824b8-ccde-4e38-bf97-87e282001b06 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:14.712466 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173824b8-ccde-4e38-bf97-87e282001b06 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:14.727014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b83c35-8232-41a5-8b13-cbac72f7f112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:14.727800 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b83c35-8232-41a5-8b13-cbac72f7f112 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:14.728207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b83c35-8232-41a5-8b13-cbac72f7f112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:14.730927 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b83c35-8232-41a5-8b13-cbac72f7f112 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:14.731734 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b83c35-8232-41a5-8b13-cbac72f7f112 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:14.733776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b83c35-8232-41a5-8b13-cbac72f7f112 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:14.742497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f7dec-248d-4343-976d-cdb8718e1942 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:14.750727 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e7a2b-6258-45fc-888b-b673d950599c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:14.751787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e7a2b-6258-45fc-888b-b673d950599c service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:14.752356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e7a2b-6258-45fc-888b-b673d950599c service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:14.752769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e7a2b-6258-45fc-888b-b673d950599c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:14.766053 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e7a2b-6258-45fc-888b-b673d950599c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:14.767933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e7a2b-6258-45fc-888b-b673d950599c service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:14.784832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 832/1666] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:06:14.810995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e7a2b-6258-45fc-888b-b673d950599c service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:14.811250 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 835/1667] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:14 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:14.826080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e7a2b-6258-45fc-888b-b673d950599c service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:15.023645 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b83c35-8232-41a5-8b13-cbac72f7f112 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:15.069760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:15.087524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2df7bc-8a78-4e35-8102-cc651699e22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:15.088992 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2df7bc-8a78-4e35-8102-cc651699e22a admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:15.090411 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2df7bc-8a78-4e35-8102-cc651699e22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:15.090703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2df7bc-8a78-4e35-8102-cc651699e22a admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:15.095543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2df7bc-8a78-4e35-8102-cc651699e22a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:15.097781 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2df7bc-8a78-4e35-8102-cc651699e22a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:15.140420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 833/1668] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:14 2022] POST /identity/v3/users => generated 294 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:15.141959 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2df7bc-8a78-4e35-8102-cc651699e22a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:15.157220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2df7bc-8a78-4e35-8102-cc651699e22a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:15.160676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b83c35-8232-41a5-8b13-cbac72f7f112 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:15.163524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f5764-009e-40a3-bece-10538de05fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:15.163524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f5764-009e-40a3-bece-10538de05fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:15.164164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f5764-009e-40a3-bece-10538de05fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:15.164446 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f5764-009e-40a3-bece-10538de05fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:15.165154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f5764-009e-40a3-bece-10538de05fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:15.167419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f5764-009e-40a3-bece-10538de05fd7 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:15.172445 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a062bdc-1002-4f1d-985e-5b6f40e4824b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:15.173483 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a062bdc-1002-4f1d-985e-5b6f40e4824b service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:15.174721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a062bdc-1002-4f1d-985e-5b6f40e4824b service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:15.175008 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a062bdc-1002-4f1d-985e-5b6f40e4824b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:15.181096 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2a062bdc-1002-4f1d-985e-5b6f40e4824b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:15.185805 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 834/1669] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:15.186766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f5764-009e-40a3-bece-10538de05fd7 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:15.204790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f5764-009e-40a3-bece-10538de05fd7 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:15.216172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b5d45-8b45-443c-8591-42f6dcf1abf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:15.217391 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b5d45-8b45-443c-8591-42f6dcf1abf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:15.218000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b5d45-8b45-443c-8591-42f6dcf1abf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:15.218541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b5d45-8b45-443c-8591-42f6dcf1abf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:15.229273 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b5d45-8b45-443c-8591-42f6dcf1abf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:15.233222 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b5d45-8b45-443c-8591-42f6dcf1abf6 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:15.262916 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 835/1670] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:15.263890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b5d45-8b45-443c-8591-42f6dcf1abf6 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:15.271963 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:15.278307 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b5d45-8b45-443c-8591-42f6dcf1abf6 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:15.285158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969718ae-5655-40b8-bd13-2eb487ae8752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:15.286574 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969718ae-5655-40b8-bd13-2eb487ae8752 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:15.287210 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969718ae-5655-40b8-bd13-2eb487ae8752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:15.287697 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969718ae-5655-40b8-bd13-2eb487ae8752 admin admin] PATH_INFO: `/v3/projects/e7453fe81a1940eb8542c53c1aefedc7/users/04b5be9fab4149c98db2e40d8f475141/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:15.315452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969718ae-5655-40b8-bd13-2eb487ae8752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7453fe81a1940eb8542c53c1aefedc7, user_id=04b5be9fab4149c98db2e40d8f475141, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:15.323759 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969718ae-5655-40b8-bd13-2eb487ae8752 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:15.330691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a062bdc-1002-4f1d-985e-5b6f40e4824b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:15.332326 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a062bdc-1002-4f1d-985e-5b6f40e4824b service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:15.348381 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 836/1672] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:15.357998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 836/1672] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:15 2022] PUT /identity/v3/projects/e7453fe81a1940eb8542c53c1aefedc7/users/04b5be9fab4149c98db2e40d8f475141/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:15.358283 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f1c62-8885-4eed-a52a-451a6f31c987 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:15.358462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f1c62-8885-4eed-a52a-451a6f31c987 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:15.358740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f1c62-8885-4eed-a52a-451a6f31c987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:15.394581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-003f1c62-8885-4eed-a52a-451a6f31c987 None None] MFA Rules not processed for user `04b5be9fab4149c98db2e40d8f475141`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:15.397046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-003f1c62-8885-4eed-a52a-451a6f31c987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:15.511510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969718ae-5655-40b8-bd13-2eb487ae8752 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:15.523918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 837/1673] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:06:15.619397 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969718ae-5655-40b8-bd13-2eb487ae8752 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:15.653479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f4f68-3c8e-4888-9845-0fc127571658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:15.654189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f4f68-3c8e-4888-9845-0fc127571658 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:15.655088 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f4f68-3c8e-4888-9845-0fc127571658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:15.655088 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f4f68-3c8e-4888-9845-0fc127571658 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:15.666860 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f4f68-3c8e-4888-9845-0fc127571658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:15.669783 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f4f68-3c8e-4888-9845-0fc127571658 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:15.733849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 837/1674] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:15 2022] POST /identity/v3/projects => generated 385 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:15.738810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f1c62-8885-4eed-a52a-451a6f31c987 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:15.768223 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f1c62-8885-4eed-a52a-451a6f31c987 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:15.976902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0f0b8-9c63-4038-9a47-ee79c2ccb143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:15.977880 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0f0b8-9c63-4038-9a47-ee79c2ccb143 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:15.978538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0f0b8-9c63-4038-9a47-ee79c2ccb143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:15.979478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0f0b8-9c63-4038-9a47-ee79c2ccb143 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:15.980448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0f0b8-9c63-4038-9a47-ee79c2ccb143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:16.011304 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0f0b8-9c63-4038-9a47-ee79c2ccb143 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:16.036807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:16.073836 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 838/1675] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:15 2022] POST /identity/v3/users => generated 299 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:16.075146 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f4f68-3c8e-4888-9845-0fc127571658 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:16.107798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f4f68-3c8e-4888-9845-0fc127571658 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:16.124055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07736038-d223-4146-a44e-cc1798925d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:16.125077 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07736038-d223-4146-a44e-cc1798925d68 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:16.125591 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07736038-d223-4146-a44e-cc1798925d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:16.125977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07736038-d223-4146-a44e-cc1798925d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:16.129336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07736038-d223-4146-a44e-cc1798925d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:16.138433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07736038-d223-4146-a44e-cc1798925d68 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:16.203123 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 838/1676] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:16.207477 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0f0b8-9c63-4038-9a47-ee79c2ccb143 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:16.250642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0f0b8-9c63-4038-9a47-ee79c2ccb143 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:16.266460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0b250-3bee-45bd-87a4-8fb18d7584dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:16.266789 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0b250-3bee-45bd-87a4-8fb18d7584dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:16.267069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0b250-3bee-45bd-87a4-8fb18d7584dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:16.267341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0b250-3bee-45bd-87a4-8fb18d7584dc admin admin] PATH_INFO: `/v3/projects/0dd0c3afb4a74b818c8480d1608fc20d/users/cfda4a70d00948b9aa5684a883e50f42/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:16.288998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0b250-3bee-45bd-87a4-8fb18d7584dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd0c3afb4a74b818c8480d1608fc20d, user_id=cfda4a70d00948b9aa5684a883e50f42, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:16.292247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0b250-3bee-45bd-87a4-8fb18d7584dc admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:16.329481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 839/1677] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:16 2022] PUT /identity/v3/projects/0dd0c3afb4a74b818c8480d1608fc20d/users/cfda4a70d00948b9aa5684a883e50f42/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:16.333133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07736038-d223-4146-a44e-cc1798925d68 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:16.357419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07736038-d223-4146-a44e-cc1798925d68 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:16.570022 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249ae46-f47f-48c9-8944-85ad45c3f77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:16.570672 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249ae46-f47f-48c9-8944-85ad45c3f77f admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:16.570946 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249ae46-f47f-48c9-8944-85ad45c3f77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:16.571222 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249ae46-f47f-48c9-8944-85ad45c3f77f admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:16.571860 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249ae46-f47f-48c9-8944-85ad45c3f77f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:16.573500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249ae46-f47f-48c9-8944-85ad45c3f77f admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:16.612530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0b250-3bee-45bd-87a4-8fb18d7584dc admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:16.637620 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0b250-3bee-45bd-87a4-8fb18d7584dc admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:16.639700 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 839/1678] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:16.656704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989226a-97bb-40a5-830c-05a09d7b2a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:16.659599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989226a-97bb-40a5-830c-05a09d7b2a13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:16.660045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989226a-97bb-40a5-830c-05a09d7b2a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:16.660475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989226a-97bb-40a5-830c-05a09d7b2a13 admin admin] PATH_INFO: `/v3/projects/0dd0c3afb4a74b818c8480d1608fc20d/users/cfda4a70d00948b9aa5684a883e50f42/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:16.664499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989226a-97bb-40a5-830c-05a09d7b2a13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd0c3afb4a74b818c8480d1608fc20d, user_id=cfda4a70d00948b9aa5684a883e50f42, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:16.701824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989226a-97bb-40a5-830c-05a09d7b2a13 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:16.759348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf89e1-7aea-4b0f-b3cb-9be7bb86ffc7 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:16.759953 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf89e1-7aea-4b0f-b3cb-9be7bb86ffc7 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:16.760220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf89e1-7aea-4b0f-b3cb-9be7bb86ffc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:16.764954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 840/1679] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:16 2022] PUT /identity/v3/projects/0dd0c3afb4a74b818c8480d1608fc20d/users/cfda4a70d00948b9aa5684a883e50f42/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:16.805126 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e023b5-754b-4f8b-8d45-e146bc299cb2 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:16.805409 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e023b5-754b-4f8b-8d45-e146bc299cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:16.805683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e023b5-754b-4f8b-8d45-e146bc299cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:16.856998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-abbf89e1-7aea-4b0f-b3cb-9be7bb86ffc7 None None] MFA Rules not processed for user `cfda4a70d00948b9aa5684a883e50f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:16.858279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-abbf89e1-7aea-4b0f-b3cb-9be7bb86ffc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:16.895555 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-75e023b5-754b-4f8b-8d45-e146bc299cb2 None None] MFA Rules not processed for user `326c59915e3d4b7289472d1400730a07`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:16.897221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-75e023b5-754b-4f8b-8d45-e146bc299cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:17.073286 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:17.175847 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 841/1680] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:17.178262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 840/1681] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:17.188192 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13604d9e-5d00-4f7e-8b6d-340600d1151c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:17.188484 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13604d9e-5d00-4f7e-8b6d-340600d1151c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:17.188760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13604d9e-5d00-4f7e-8b6d-340600d1151c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:17.204277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf89e1-7aea-4b0f-b3cb-9be7bb86ffc7 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:17.206783 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-abbf89e1-7aea-4b0f-b3cb-9be7bb86ffc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:17.267494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-13604d9e-5d00-4f7e-8b6d-340600d1151c None None] MFA Rules not processed for user `326c59915e3d4b7289472d1400730a07`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:17.268951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-13604d9e-5d00-4f7e-8b6d-340600d1151c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:17.303171 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 842/1682] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:17.313155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13604d9e-5d00-4f7e-8b6d-340600d1151c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:17.314945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-13604d9e-5d00-4f7e-8b6d-340600d1151c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:17.413148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf89e1-7aea-4b0f-b3cb-9be7bb86ffc7 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:17.464464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89b330-d790-4dc3-b52a-a717d12bb384 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:17.465114 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89b330-d790-4dc3-b52a-a717d12bb384 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:17.465392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89b330-d790-4dc3-b52a-a717d12bb384 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:17.465662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89b330-d790-4dc3-b52a-a717d12bb384 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:17.501616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89b330-d790-4dc3-b52a-a717d12bb384 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:17.503610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89b330-d790-4dc3-b52a-a717d12bb384 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:17.535765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 841/1683] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:17.578222 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13604d9e-5d00-4f7e-8b6d-340600d1151c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:17.629083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc779d-e6f0-4560-bdd5-8073b16664f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:17.630073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc779d-e6f0-4560-bdd5-8073b16664f1 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:17.630073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc779d-e6f0-4560-bdd5-8073b16664f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:17.634602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc779d-e6f0-4560-bdd5-8073b16664f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:17.677951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc779d-e6f0-4560-bdd5-8073b16664f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:17.679638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc779d-e6f0-4560-bdd5-8073b16664f1 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:17.710875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89b330-d790-4dc3-b52a-a717d12bb384 service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:17.737861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 843/1684] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:06:17.901494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89b330-d790-4dc3-b52a-a717d12bb384 service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:17.921171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da625b13-e316-4a5a-8698-1a924e0e0ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:17.924058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da625b13-e316-4a5a-8698-1a924e0e0ca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:17.925943 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da625b13-e316-4a5a-8698-1a924e0e0ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:17.926999 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da625b13-e316-4a5a-8698-1a924e0e0ca1 admin admin] PATH_INFO: `/v3/users/04b5be9fab4149c98db2e40d8f475141` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:17.939448 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da625b13-e316-4a5a-8698-1a924e0e0ca1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04b5be9fab4149c98db2e40d8f475141)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:17.949195 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da625b13-e316-4a5a-8698-1a924e0e0ca1 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:18.172203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-da625b13-e316-4a5a-8698-1a924e0e0ca1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04b5be9fab4149c98db2e40d8f475141'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:18.201736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-da625b13-e316-4a5a-8698-1a924e0e0ca1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04b5be9fab4149c98db2e40d8f475141'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:18.271478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-da625b13-e316-4a5a-8698-1a924e0e0ca1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04b5be9fab4149c98db2e40d8f475141'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:18.300962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-da625b13-e316-4a5a-8698-1a924e0e0ca1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04b5be9fab4149c98db2e40d8f475141'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:18.314915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-da625b13-e316-4a5a-8698-1a924e0e0ca1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04b5be9fab4149c98db2e40d8f475141'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:18.317500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:18.324425 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 842/1685] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:06:17 2022] DELETE /identity/v3/users/04b5be9fab4149c98db2e40d8f475141 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:18.329210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13a8a5-2c67-430a-a239-f2a9d6edfe1f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:18.329492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13a8a5-2c67-430a-a239-f2a9d6edfe1f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:18.330144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13a8a5-2c67-430a-a239-f2a9d6edfe1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:18.384399 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0632ae-7ac4-4c15-889d-e3f7eab20f47 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:18.384526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0632ae-7ac4-4c15-889d-e3f7eab20f47 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:18.385347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0632ae-7ac4-4c15-889d-e3f7eab20f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:18.389324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4c13a8a5-2c67-430a-a239-f2a9d6edfe1f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:18.390424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4c13a8a5-2c67-430a-a239-f2a9d6edfe1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:18.428675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:18.463598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-bc0632ae-7ac4-4c15-889d-e3f7eab20f47 None None] MFA Rules not processed for user `9c0f1bfeb1464f87a7f3ea4add940861`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:18.467197 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-bc0632ae-7ac4-4c15-889d-e3f7eab20f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:18.576594 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 844/1686] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:18.588487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13a8a5-2c67-430a-a239-f2a9d6edfe1f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:18.594146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4c13a8a5-2c67-430a-a239-f2a9d6edfe1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:18.666444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 843/1687] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:18.670367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74410bb-0e12-4f23-b60d-aaee0a510e5b None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:18.670661 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74410bb-0e12-4f23-b60d-aaee0a510e5b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:18.673825 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74410bb-0e12-4f23-b60d-aaee0a510e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:18.728090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a74410bb-0e12-4f23-b60d-aaee0a510e5b None None] MFA Rules not processed for user `9c0f1bfeb1464f87a7f3ea4add940861`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:18.734738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a74410bb-0e12-4f23-b60d-aaee0a510e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:18.756253 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 844/1688] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:18.763847 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13a8a5-2c67-430a-a239-f2a9d6edfe1f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:18.768156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74410bb-0e12-4f23-b60d-aaee0a510e5b None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:18.769786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb23549-11b7-4946-a9b0-6de83634bd8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:18.770375 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a74410bb-0e12-4f23-b60d-aaee0a510e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:18.770646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb23549-11b7-4946-a9b0-6de83634bd8e service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:18.771048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb23549-11b7-4946-a9b0-6de83634bd8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:18.771432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb23549-11b7-4946-a9b0-6de83634bd8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:18.798252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb23549-11b7-4946-a9b0-6de83634bd8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:18.800025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb23549-11b7-4946-a9b0-6de83634bd8e service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:18.820834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 845/1689] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:06:18.961151 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74410bb-0e12-4f23-b60d-aaee0a510e5b None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:18.980344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003494ef-e906-4815-88f2-478b91568276 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:18.981028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003494ef-e906-4815-88f2-478b91568276 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:18.981306 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003494ef-e906-4815-88f2-478b91568276 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:18.981573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003494ef-e906-4815-88f2-478b91568276 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:19.001969 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003494ef-e906-4815-88f2-478b91568276 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:19.004705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003494ef-e906-4815-88f2-478b91568276 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:19.018529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 845/1690] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:18 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:19.304108 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb23549-11b7-4946-a9b0-6de83634bd8e service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:19.305899 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6cb23549-11b7-4946-a9b0-6de83634bd8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:19.351807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb23549-11b7-4946-a9b0-6de83634bd8e service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:19.367590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f460bc5-e5df-40c4-86b8-ca8836d4bd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:19.368676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f460bc5-e5df-40c4-86b8-ca8836d4bd9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:19.369974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f460bc5-e5df-40c4-86b8-ca8836d4bd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:19.370508 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f460bc5-e5df-40c4-86b8-ca8836d4bd9b admin admin] PATH_INFO: `/v3/projects/e7453fe81a1940eb8542c53c1aefedc7` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:19.373991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f460bc5-e5df-40c4-86b8-ca8836d4bd9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7453fe81a1940eb8542c53c1aefedc7)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:19.377816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f460bc5-e5df-40c4-86b8-ca8836d4bd9b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:19.480615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7f460bc5-e5df-40c4-86b8-ca8836d4bd9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7453fe81a1940eb8542c53c1aefedc7', 'request_id': 'req-7f460bc5-e5df-40c4-86b8-ca8836d4bd9b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:19.496034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7f460bc5-e5df-40c4-86b8-ca8836d4bd9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7453fe81a1940eb8542c53c1aefedc7', 'request_id': 'req-7f460bc5-e5df-40c4-86b8-ca8836d4bd9b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:19.505661 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7f460bc5-e5df-40c4-86b8-ca8836d4bd9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7453fe81a1940eb8542c53c1aefedc7', 'request_id': 'req-7f460bc5-e5df-40c4-86b8-ca8836d4bd9b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:19.525139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7f460bc5-e5df-40c4-86b8-ca8836d4bd9b admin admin] The token cache is being invalidate because project e7453fe81a1940eb8542c53c1aefedc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:06:19.525679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7f460bc5-e5df-40c4-86b8-ca8836d4bd9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:19.526862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7f460bc5-e5df-40c4-86b8-ca8836d4bd9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:19.533763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 846/1691] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:06:19 2022] DELETE /identity/v3/projects/e7453fe81a1940eb8542c53c1aefedc7 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:19.540810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003494ef-e906-4815-88f2-478b91568276 service nova] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:19.626405 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003494ef-e906-4815-88f2-478b91568276 service nova] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:19.752439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b4dad-3281-4e48-b403-064e3faef926 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:19.752754 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b4dad-3281-4e48-b403-064e3faef926 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:19.753027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b4dad-3281-4e48-b403-064e3faef926 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:19.757136 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 847/1692] 10.222.0.62 () {54 vars in 912 bytes} [Fri Sep 9 12:06:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:19.762412 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15cc86-514a-4ee7-809e-62b8d0d3a8ec None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:19.762699 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15cc86-514a-4ee7-809e-62b8d0d3a8ec None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:19.762980 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15cc86-514a-4ee7-809e-62b8d0d3a8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:19.834826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34d25c-d308-4643-91f2-5377dd5bdfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:19.835657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34d25c-d308-4643-91f2-5377dd5bdfe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:19.835936 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34d25c-d308-4643-91f2-5377dd5bdfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:19.836201 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34d25c-d308-4643-91f2-5377dd5bdfe4 admin admin] PATH_INFO: `/v3/users/cfda4a70d00948b9aa5684a883e50f42` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:19.837411 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34d25c-d308-4643-91f2-5377dd5bdfe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfda4a70d00948b9aa5684a883e50f42)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:19.844037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34d25c-d308-4643-91f2-5377dd5bdfe4 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:19.874368 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:19.903780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-bc15cc86-514a-4ee7-809e-62b8d0d3a8ec None None] MFA Rules not processed for user `d00634cdd3fb41e794d3af0b88ba7338`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:19.905846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bc15cc86-514a-4ee7-809e-62b8d0d3a8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:19.966516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-cf34d25c-d308-4643-91f2-5377dd5bdfe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfda4a70d00948b9aa5684a883e50f42'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:19.981800 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-cf34d25c-d308-4643-91f2-5377dd5bdfe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfda4a70d00948b9aa5684a883e50f42'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:19.985380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:20.010891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-cf34d25c-d308-4643-91f2-5377dd5bdfe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfda4a70d00948b9aa5684a883e50f42'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:20.024260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-cf34d25c-d308-4643-91f2-5377dd5bdfe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfda4a70d00948b9aa5684a883e50f42'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:20.025068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-cf34d25c-d308-4643-91f2-5377dd5bdfe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfda4a70d00948b9aa5684a883e50f42'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:20.032493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 846/1693] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:06:19 2022] DELETE /identity/v3/users/cfda4a70d00948b9aa5684a883e50f42 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:20.044784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 848/1694] 10.222.0.62 () {58 vars in 1009 bytes} [Fri Sep 9 12:06:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 09 12:06:20.061778 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34d25c-d308-4643-91f2-5377dd5bdfe4 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:20.063513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-cf34d25c-d308-4643-91f2-5377dd5bdfe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:20.068228 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc84b4-c911-4cd9-a531-cf6acdbaf62f None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:20.068514 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc84b4-c911-4cd9-a531-cf6acdbaf62f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:20.068788 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc84b4-c911-4cd9-a531-cf6acdbaf62f None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:20.070733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 849/1695] 10.222.0.62 () {54 vars in 912 bytes} [Fri Sep 9 12:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:06:20.082890 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e8917-34c4-4418-aefb-30e6411d88ab None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:20.083279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e8917-34c4-4418-aefb-30e6411d88ab None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:20.083709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e8917-34c4-4418-aefb-30e6411d88ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:20.111860 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-662e8917-34c4-4418-aefb-30e6411d88ab None None] MFA Rules not processed for user `d00634cdd3fb41e794d3af0b88ba7338`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:20.113447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-662e8917-34c4-4418-aefb-30e6411d88ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:20.191219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 850/1696] 10.222.0.62 () {58 vars in 1009 bytes} [Fri Sep 9 12:06:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:20.200907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34d25c-d308-4643-91f2-5377dd5bdfe4 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:20.207443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65b47a-ee13-42af-968a-820be9a81e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:20.208720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65b47a-ee13-42af-968a-820be9a81e0b service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:20.209231 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65b47a-ee13-42af-968a-820be9a81e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:20.209691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65b47a-ee13-42af-968a-820be9a81e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:20.224780 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65b47a-ee13-42af-968a-820be9a81e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:20.228226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65b47a-ee13-42af-968a-820be9a81e0b service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:20.243245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 847/1697] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:20.505388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e8917-34c4-4418-aefb-30e6411d88ab None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:20.508097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-662e8917-34c4-4418-aefb-30e6411d88ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:20.532275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e8917-34c4-4418-aefb-30e6411d88ab None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:20.541535 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890fdeb-814e-4190-8109-5f01aed58fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:20.544043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890fdeb-814e-4190-8109-5f01aed58fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:20.544196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890fdeb-814e-4190-8109-5f01aed58fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:20.544521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890fdeb-814e-4190-8109-5f01aed58fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:20.567997 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890fdeb-814e-4190-8109-5f01aed58fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:20.573229 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890fdeb-814e-4190-8109-5f01aed58fb0 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:20.595958 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 851/1698] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:20.706100 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65b47a-ee13-42af-968a-820be9a81e0b service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:20.708144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0f65b47a-ee13-42af-968a-820be9a81e0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:20.884307 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65b47a-ee13-42af-968a-820be9a81e0b service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:20.893509 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926ad42-ebc6-473d-880d-85092ba97ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:20.894605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926ad42-ebc6-473d-880d-85092ba97ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:20.895088 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926ad42-ebc6-473d-880d-85092ba97ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:20.895598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926ad42-ebc6-473d-880d-85092ba97ed1 admin admin] PATH_INFO: `/v3/projects/0dd0c3afb4a74b818c8480d1608fc20d` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:20.897135 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926ad42-ebc6-473d-880d-85092ba97ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dd0c3afb4a74b818c8480d1608fc20d)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:20.899998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926ad42-ebc6-473d-880d-85092ba97ed1 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:20.988775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:21.031084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4926ad42-ebc6-473d-880d-85092ba97ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dd0c3afb4a74b818c8480d1608fc20d', 'request_id': 'req-4926ad42-ebc6-473d-880d-85092ba97ed1'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:21.051705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4926ad42-ebc6-473d-880d-85092ba97ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dd0c3afb4a74b818c8480d1608fc20d', 'request_id': 'req-4926ad42-ebc6-473d-880d-85092ba97ed1'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:21.079935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4926ad42-ebc6-473d-880d-85092ba97ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dd0c3afb4a74b818c8480d1608fc20d', 'request_id': 'req-4926ad42-ebc6-473d-880d-85092ba97ed1'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:21.099603 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4926ad42-ebc6-473d-880d-85092ba97ed1 admin admin] The token cache is being invalidate because project 0dd0c3afb4a74b818c8480d1608fc20d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:06:21.099909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4926ad42-ebc6-473d-880d-85092ba97ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:21.100676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4926ad42-ebc6-473d-880d-85092ba97ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:21.102268 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 848/1699] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:06:20 2022] DELETE /identity/v3/projects/0dd0c3afb4a74b818c8480d1608fc20d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:06:21.107865 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d6e20-71eb-4801-a383-cbd0d7b809bc None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:21.108842 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d6e20-71eb-4801-a383-cbd0d7b809bc None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:21.109128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d6e20-71eb-4801-a383-cbd0d7b809bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:21.172556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-067d6e20-71eb-4801-a383-cbd0d7b809bc None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:21.174568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-067d6e20-71eb-4801-a383-cbd0d7b809bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:21.215326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926ad42-ebc6-473d-880d-85092ba97ed1 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:21.216808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-4926ad42-ebc6-473d-880d-85092ba97ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:21.321825 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926ad42-ebc6-473d-880d-85092ba97ed1 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:21.328430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132737f-c433-4a23-af92-8f019d88c0de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:21.329305 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132737f-c433-4a23-af92-8f019d88c0de service placement] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:21.329659 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132737f-c433-4a23-af92-8f019d88c0de service placement] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:21.330003 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132737f-c433-4a23-af92-8f019d88c0de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:21.333077 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2132737f-c433-4a23-af92-8f019d88c0de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:21.352965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132737f-c433-4a23-af92-8f019d88c0de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:21.355656 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132737f-c433-4a23-af92-8f019d88c0de service placement] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:21.364195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 852/1700] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:21.374259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171651a-33b6-493c-9524-72520b23d77f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:21.374547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171651a-33b6-493c-9524-72520b23d77f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:21.374826 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171651a-33b6-493c-9524-72520b23d77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:21.385527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 849/1701] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:21.424708 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3422ca-4fb5-4a50-93a4-b655f9833c19 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:21.424996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3422ca-4fb5-4a50-93a4-b655f9833c19 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:21.425272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3422ca-4fb5-4a50-93a4-b655f9833c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:21.443446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5171651a-33b6-493c-9524-72520b23d77f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:21.444772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5171651a-33b6-493c-9524-72520b23d77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:21.460551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-9e3422ca-4fb5-4a50-93a4-b655f9833c19 None None] MFA Rules not processed for user `9c0f1bfeb1464f87a7f3ea4add940861`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:21.461818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-9e3422ca-4fb5-4a50-93a4-b655f9833c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:21.466389 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 853/1702] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:21.470277 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171651a-33b6-493c-9524-72520b23d77f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:21.487301 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171651a-33b6-493c-9524-72520b23d77f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:21.501929 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c076a38-5681-4b76-9de3-898960c21ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:21.502653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c076a38-5681-4b76-9de3-898960c21ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:21.502933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c076a38-5681-4b76-9de3-898960c21ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:21.503207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c076a38-5681-4b76-9de3-898960c21ccf admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:21.504156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c076a38-5681-4b76-9de3-898960c21ccf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:21.504522 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c076a38-5681-4b76-9de3-898960c21ccf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:21.507688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c076a38-5681-4b76-9de3-898960c21ccf admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:21.543581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 854/1703] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:06:21.546827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c076a38-5681-4b76-9de3-898960c21ccf admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:21.568709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c076a38-5681-4b76-9de3-898960c21ccf admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:21.579732 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a37868-feab-4d4d-9f50-cbff07209be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:21.580356 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a37868-feab-4d4d-9f50-cbff07209be9 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:21.580632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a37868-feab-4d4d-9f50-cbff07209be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:21.580955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a37868-feab-4d4d-9f50-cbff07209be9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:21.581545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a37868-feab-4d4d-9f50-cbff07209be9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:21.583496 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a37868-feab-4d4d-9f50-cbff07209be9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:21.623134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 850/1704] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:06:21.655574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 855/1705] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:21 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:21.659641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3422ca-4fb5-4a50-93a4-b655f9833c19 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:21.674414 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3422ca-4fb5-4a50-93a4-b655f9833c19 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:21.813857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:21.837741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c548410-ad66-4019-bfe0-57e80227af8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:21.839927 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c548410-ad66-4019-bfe0-57e80227af8a admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:21.840213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c548410-ad66-4019-bfe0-57e80227af8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:21.840985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c548410-ad66-4019-bfe0-57e80227af8a admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:21.841231 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c548410-ad66-4019-bfe0-57e80227af8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:21.844631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c548410-ad66-4019-bfe0-57e80227af8a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:21.905314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 851/1706] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:21 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:21.909499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a37868-feab-4d4d-9f50-cbff07209be9 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:21.923230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a37868-feab-4d4d-9f50-cbff07209be9 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:21.941250 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede5e46-0f0f-4f9f-95b9-0ad678a56e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:21.941888 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede5e46-0f0f-4f9f-95b9-0ad678a56e1c admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:21.942193 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede5e46-0f0f-4f9f-95b9-0ad678a56e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:21.942467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede5e46-0f0f-4f9f-95b9-0ad678a56e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:21.943162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede5e46-0f0f-4f9f-95b9-0ad678a56e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:21.944763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede5e46-0f0f-4f9f-95b9-0ad678a56e1c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:21.974086 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 856/1707] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:21.982354 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c548410-ad66-4019-bfe0-57e80227af8a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:21.998938 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c548410-ad66-4019-bfe0-57e80227af8a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:22.007456 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf97a3-3690-43b1-ab7a-e927150385a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:22.008078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf97a3-3690-43b1-ab7a-e927150385a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:22.008350 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf97a3-3690-43b1-ab7a-e927150385a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:22.008617 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf97a3-3690-43b1-ab7a-e927150385a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:22.009267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf97a3-3690-43b1-ab7a-e927150385a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:22.011467 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf97a3-3690-43b1-ab7a-e927150385a6 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:22.039377 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 852/1708] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:22.043362 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede5e46-0f0f-4f9f-95b9-0ad678a56e1c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:22.060631 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede5e46-0f0f-4f9f-95b9-0ad678a56e1c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:22.065431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc0b46-a660-434b-8e5e-0d64a2b553df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:22.066110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc0b46-a660-434b-8e5e-0d64a2b553df admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:22.068475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc0b46-a660-434b-8e5e-0d64a2b553df admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:22.068759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc0b46-a660-434b-8e5e-0d64a2b553df admin admin] PATH_INFO: `/v3/projects/ab4dbbb8fd204c1a84635df66e1b24f2/users/8ca25551d5704ec4b432c40ad4916a3a/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:22.112592 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc0b46-a660-434b-8e5e-0d64a2b553df admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab4dbbb8fd204c1a84635df66e1b24f2, user_id=8ca25551d5704ec4b432c40ad4916a3a, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:22.117562 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc0b46-a660-434b-8e5e-0d64a2b553df admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:22.153740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 857/1709] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:22 2022] PUT /identity/v3/projects/ab4dbbb8fd204c1a84635df66e1b24f2/users/8ca25551d5704ec4b432c40ad4916a3a/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:22.164233 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04380540-c3cc-4d09-8564-be9b294c256d None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:22.164518 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04380540-c3cc-4d09-8564-be9b294c256d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:22.164787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04380540-c3cc-4d09-8564-be9b294c256d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:22.266148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-04380540-c3cc-4d09-8564-be9b294c256d None None] MFA Rules not processed for user `8ca25551d5704ec4b432c40ad4916a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:22.267510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-04380540-c3cc-4d09-8564-be9b294c256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:22.684661 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 853/1710] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 518 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:22.688292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc0b46-a660-434b-8e5e-0d64a2b553df admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:22.695023 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:22.945596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc0b46-a660-434b-8e5e-0d64a2b553df admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:22.981463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c947b83-4fc2-41e8-83ff-c15788c07af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:22.984859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c947b83-4fc2-41e8-83ff-c15788c07af6 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:22.985458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c947b83-4fc2-41e8-83ff-c15788c07af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:22.985831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c947b83-4fc2-41e8-83ff-c15788c07af6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:22.986629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c947b83-4fc2-41e8-83ff-c15788c07af6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:22.988657 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c947b83-4fc2-41e8-83ff-c15788c07af6 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:23.075166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 858/1711] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:22 2022] POST /identity/v3/projects => generated 387 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:23.076843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04380540-c3cc-4d09-8564-be9b294c256d None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:23.097483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04380540-c3cc-4d09-8564-be9b294c256d None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:23.333167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d9f16-e653-471d-8e90-c11b009b5eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:23.334533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d9f16-e653-471d-8e90-c11b009b5eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:23.334738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d9f16-e653-471d-8e90-c11b009b5eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:23.335197 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d9f16-e653-471d-8e90-c11b009b5eb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:23.336529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d9f16-e653-471d-8e90-c11b009b5eb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:23.341129 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d9f16-e653-471d-8e90-c11b009b5eb7 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:23.412720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 854/1712] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:23 2022] POST /identity/v3/users => generated 294 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:23.412931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c947b83-4fc2-41e8-83ff-c15788c07af6 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:23.441214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c947b83-4fc2-41e8-83ff-c15788c07af6 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:23.477918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cbeaf-5f02-4d47-b09b-b2374acb419d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:23.479168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cbeaf-5f02-4d47-b09b-b2374acb419d admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:23.496388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cbeaf-5f02-4d47-b09b-b2374acb419d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:23.496951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cbeaf-5f02-4d47-b09b-b2374acb419d admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:23.497675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cbeaf-5f02-4d47-b09b-b2374acb419d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:23.499336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cbeaf-5f02-4d47-b09b-b2374acb419d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:23.539838 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d9f16-e653-471d-8e90-c11b009b5eb7 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:23.541547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 859/1713] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:23.555689 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d9f16-e653-471d-8e90-c11b009b5eb7 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:23.586887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4adbb-eeb5-4440-b75b-a80432deb2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:23.589272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4adbb-eeb5-4440-b75b-a80432deb2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:23.590790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4adbb-eeb5-4440-b75b-a80432deb2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:23.593251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4adbb-eeb5-4440-b75b-a80432deb2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:23.593979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4adbb-eeb5-4440-b75b-a80432deb2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:23.604050 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4adbb-eeb5-4440-b75b-a80432deb2c3 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:23.634478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 855/1714] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:23.638470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cbeaf-5f02-4d47-b09b-b2374acb419d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:23.654930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cbeaf-5f02-4d47-b09b-b2374acb419d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:23.666643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b28a5-bb0e-4b1d-a8c5-74b36dac3fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:23.667798 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b28a5-bb0e-4b1d-a8c5-74b36dac3fc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:23.672488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b28a5-bb0e-4b1d-a8c5-74b36dac3fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:23.672951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b28a5-bb0e-4b1d-a8c5-74b36dac3fc6 admin admin] PATH_INFO: `/v3/projects/26a6bdd442d648a29637e52798ba11ff/users/61c303082a4845889916f0cfcbc57e37/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:23.742534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b28a5-bb0e-4b1d-a8c5-74b36dac3fc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a6bdd442d648a29637e52798ba11ff, user_id=61c303082a4845889916f0cfcbc57e37, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:23.754879 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b28a5-bb0e-4b1d-a8c5-74b36dac3fc6 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:23.798260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 860/1715] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:23 2022] PUT /identity/v3/projects/26a6bdd442d648a29637e52798ba11ff/users/61c303082a4845889916f0cfcbc57e37/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:23.809737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892b6e5-ebe5-453e-b74b-f604097de0d6 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:23.809931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892b6e5-ebe5-453e-b74b-f604097de0d6 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:23.810175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892b6e5-ebe5-453e-b74b-f604097de0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:23.887111 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-4892b6e5-ebe5-453e-b74b-f604097de0d6 None None] MFA Rules not processed for user `61c303082a4845889916f0cfcbc57e37`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:23.888376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-4892b6e5-ebe5-453e-b74b-f604097de0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:23.977856 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:24.106356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 856/1716] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:24.108540 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b28a5-bb0e-4b1d-a8c5-74b36dac3fc6 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:24.317155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b28a5-bb0e-4b1d-a8c5-74b36dac3fc6 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:24.323482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18c54c-a7bc-47bd-8f56-eaeba9b79b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:24.324320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18c54c-a7bc-47bd-8f56-eaeba9b79b16 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:24.324678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18c54c-a7bc-47bd-8f56-eaeba9b79b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:24.325028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18c54c-a7bc-47bd-8f56-eaeba9b79b16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:24.325849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18c54c-a7bc-47bd-8f56-eaeba9b79b16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:24.329191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18c54c-a7bc-47bd-8f56-eaeba9b79b16 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:24.381508 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 861/1717] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:24 2022] POST /identity/v3/projects => generated 385 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:24.386634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892b6e5-ebe5-453e-b74b-f604097de0d6 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:24.407113 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892b6e5-ebe5-453e-b74b-f604097de0d6 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:24.628083 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7289db-2910-42e9-ad34-a042dcf93585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:24.628954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7289db-2910-42e9-ad34-a042dcf93585 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:24.629158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7289db-2910-42e9-ad34-a042dcf93585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:24.629433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7289db-2910-42e9-ad34-a042dcf93585 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:24.630101 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7289db-2910-42e9-ad34-a042dcf93585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:24.632127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7289db-2910-42e9-ad34-a042dcf93585 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:24.700525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18c54c-a7bc-47bd-8f56-eaeba9b79b16 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:24.711469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 857/1718] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:24 2022] POST /identity/v3/users => generated 299 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:24.714729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18c54c-a7bc-47bd-8f56-eaeba9b79b16 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:24.738730 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b05be-90e6-446e-99de-3647ea336efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:24.740148 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b05be-90e6-446e-99de-3647ea336efc admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:24.740437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b05be-90e6-446e-99de-3647ea336efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:24.741836 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b05be-90e6-446e-99de-3647ea336efc admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:24.742553 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b05be-90e6-446e-99de-3647ea336efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:24.745794 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b05be-90e6-446e-99de-3647ea336efc admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:24.766844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 862/1719] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:24.771895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7289db-2910-42e9-ad34-a042dcf93585 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:24.789593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7289db-2910-42e9-ad34-a042dcf93585 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:24.809301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c6616-215e-4b1a-bd6e-779781440aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:24.810347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c6616-215e-4b1a-bd6e-779781440aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:24.810787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c6616-215e-4b1a-bd6e-779781440aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:24.811215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c6616-215e-4b1a-bd6e-779781440aaf admin admin] PATH_INFO: `/v3/projects/dfe5946c69b7499fbdfb8991d65729a2/users/ee00b721f06f4aacaaf83a6a96b67c4d/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:24.837044 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c6616-215e-4b1a-bd6e-779781440aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfe5946c69b7499fbdfb8991d65729a2, user_id=ee00b721f06f4aacaaf83a6a96b67c4d, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:24.842029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c6616-215e-4b1a-bd6e-779781440aaf admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:24.883022 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 858/1720] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:24 2022] PUT /identity/v3/projects/dfe5946c69b7499fbdfb8991d65729a2/users/ee00b721f06f4aacaaf83a6a96b67c4d/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:24.884399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b05be-90e6-446e-99de-3647ea336efc admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:24.908343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b05be-90e6-446e-99de-3647ea336efc admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:24.972434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:25.044978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6f4d9-3224-4d28-b91b-e0d604580098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:25.045892 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6f4d9-3224-4d28-b91b-e0d604580098 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:25.046412 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6f4d9-3224-4d28-b91b-e0d604580098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:25.046846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6f4d9-3224-4d28-b91b-e0d604580098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:25.047698 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6f4d9-3224-4d28-b91b-e0d604580098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:25.050230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6f4d9-3224-4d28-b91b-e0d604580098 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:25.074758 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 863/1721] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:25.079129 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c6616-215e-4b1a-bd6e-779781440aaf admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:25.100184 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c6616-215e-4b1a-bd6e-779781440aaf admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:25.122557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180584a-366f-4775-b8ce-95d211499331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:25.123411 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180584a-366f-4775-b8ce-95d211499331 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:25.123828 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180584a-366f-4775-b8ce-95d211499331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:25.124562 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180584a-366f-4775-b8ce-95d211499331 admin admin] PATH_INFO: `/v3/projects/dfe5946c69b7499fbdfb8991d65729a2/users/ee00b721f06f4aacaaf83a6a96b67c4d/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:25.135873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180584a-366f-4775-b8ce-95d211499331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfe5946c69b7499fbdfb8991d65729a2, user_id=ee00b721f06f4aacaaf83a6a96b67c4d, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:25.138657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180584a-366f-4775-b8ce-95d211499331 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:25.184624 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 859/1722] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:25 2022] PUT /identity/v3/projects/dfe5946c69b7499fbdfb8991d65729a2/users/ee00b721f06f4aacaaf83a6a96b67c4d/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:25.201305 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53ac87-ba22-4df7-8403-c77d05c2fa4c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:25.201561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53ac87-ba22-4df7-8403-c77d05c2fa4c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:25.201735 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53ac87-ba22-4df7-8403-c77d05c2fa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:25.276797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-fd53ac87-ba22-4df7-8403-c77d05c2fa4c None None] MFA Rules not processed for user `ee00b721f06f4aacaaf83a6a96b67c4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:25.278054 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fd53ac87-ba22-4df7-8403-c77d05c2fa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:25.470244 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 864/1723] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:25.477514 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180584a-366f-4775-b8ce-95d211499331 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:25.480027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-9180584a-366f-4775-b8ce-95d211499331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:25.649004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180584a-366f-4775-b8ce-95d211499331 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:25.667321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7645cd0-c98d-4e0f-a76e-8c62290dda3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:25.670848 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7645cd0-c98d-4e0f-a76e-8c62290dda3d service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:25.672031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7645cd0-c98d-4e0f-a76e-8c62290dda3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:25.673551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7645cd0-c98d-4e0f-a76e-8c62290dda3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:25.715932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7645cd0-c98d-4e0f-a76e-8c62290dda3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:25.718264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7645cd0-c98d-4e0f-a76e-8c62290dda3d service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:25.745305 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 860/1724] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:26.933686 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53ac87-ba22-4df7-8403-c77d05c2fa4c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:26.935811 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fd53ac87-ba22-4df7-8403-c77d05c2fa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:26.960300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53ac87-ba22-4df7-8403-c77d05c2fa4c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:26.980465 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0a8e0-8a93-41f4-904e-1d978f81ec7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:26.981752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0a8e0-8a93-41f4-904e-1d978f81ec7b service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:26.981752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0a8e0-8a93-41f4-904e-1d978f81ec7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:26.982267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0a8e0-8a93-41f4-904e-1d978f81ec7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:27.139968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0a8e0-8a93-41f4-904e-1d978f81ec7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:27.144229 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0a8e0-8a93-41f4-904e-1d978f81ec7b service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:27.170306 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 865/1725] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:28.125383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196ec36-7578-4004-940d-4b490bf06fde None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:28.126951 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196ec36-7578-4004-940d-4b490bf06fde None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:28.127549 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196ec36-7578-4004-940d-4b490bf06fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:28.334272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:28.405263 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-6196ec36-7578-4004-940d-4b490bf06fde None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:28.413560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6196ec36-7578-4004-940d-4b490bf06fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:28.592224 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 861/1726] 10.222.0.62 () {58 vars in 1009 bytes} [Fri Sep 9 12:06:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:28.957295 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0a8e0-8a93-41f4-904e-1d978f81ec7b service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:28.959025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-dfb0a8e0-8a93-41f4-904e-1d978f81ec7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:29.149184 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:29.271936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0a8e0-8a93-41f4-904e-1d978f81ec7b service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:29.294000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4d7a6-bcf5-4649-b09b-a6d90ba53652 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:29.295128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4d7a6-bcf5-4649-b09b-a6d90ba53652 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:29.295949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4d7a6-bcf5-4649-b09b-a6d90ba53652 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:29.296367 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4d7a6-bcf5-4649-b09b-a6d90ba53652 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:29.324796 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4d7a6-bcf5-4649-b09b-a6d90ba53652 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:29.326435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4d7a6-bcf5-4649-b09b-a6d90ba53652 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:29.335782 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 866/1727] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:06:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:06:29.583252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f258ee-334c-48d2-85b0-7724ef225d41 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:29.583732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f258ee-334c-48d2-85b0-7724ef225d41 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:29.583809 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f258ee-334c-48d2-85b0-7724ef225d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:29.685034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-24f258ee-334c-48d2-85b0-7724ef225d41 None None] MFA Rules not processed for user `9676597a1b884138bf5435f3b6cd63c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:29.689131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-24f258ee-334c-48d2-85b0-7724ef225d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:29.894539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 862/1728] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:29.914000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4d7a6-bcf5-4649-b09b-a6d90ba53652 service swift] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:29.914000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-77e4d7a6-bcf5-4649-b09b-a6d90ba53652 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:30.227785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4d7a6-bcf5-4649-b09b-a6d90ba53652 service swift] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:30.254872 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33908281-7370-42bf-83fb-59296fd1673f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:30.255717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33908281-7370-42bf-83fb-59296fd1673f service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:30.256152 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33908281-7370-42bf-83fb-59296fd1673f service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:30.256576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33908281-7370-42bf-83fb-59296fd1673f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:30.284564 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33908281-7370-42bf-83fb-59296fd1673f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:30.286556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33908281-7370-42bf-83fb-59296fd1673f service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:30.323531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 867/1729] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:30.621758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaaa314-86bc-4f9e-8dbd-a125f75d05d8 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:30.621758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaaa314-86bc-4f9e-8dbd-a125f75d05d8 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:30.622448 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaaa314-86bc-4f9e-8dbd-a125f75d05d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:30.758935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-2eaaa314-86bc-4f9e-8dbd-a125f75d05d8 None None] MFA Rules not processed for user `bfc29d15a7344754aa53d55379e2751f`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:30.769087 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2eaaa314-86bc-4f9e-8dbd-a125f75d05d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:30.789523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33908281-7370-42bf-83fb-59296fd1673f service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:30.791852 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-33908281-7370-42bf-83fb-59296fd1673f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:30.985109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 863/1730] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:30.986962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaaa314-86bc-4f9e-8dbd-a125f75d05d8 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:31.031698 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaaa314-86bc-4f9e-8dbd-a125f75d05d8 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:31.040416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33908281-7370-42bf-83fb-59296fd1673f service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:31.059126 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f97f2-70bd-41d6-9e3c-e8d9b5817b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:31.060169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f97f2-70bd-41d6-9e3c-e8d9b5817b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:31.060665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f97f2-70bd-41d6-9e3c-e8d9b5817b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:31.061167 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f97f2-70bd-41d6-9e3c-e8d9b5817b6c admin admin] PATH_INFO: `/v3/users/9af5557669984780a427691d752922a1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:31.066072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f97f2-70bd-41d6-9e3c-e8d9b5817b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af5557669984780a427691d752922a1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:31.066279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd5d18-a52c-48f2-a191-3f723bf01235 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:31.066915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd5d18-a52c-48f2-a191-3f723bf01235 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:31.067201 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd5d18-a52c-48f2-a191-3f723bf01235 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:31.067473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd5d18-a52c-48f2-a191-3f723bf01235 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:31.069750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f97f2-70bd-41d6-9e3c-e8d9b5817b6c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:31.097657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd5d18-a52c-48f2-a191-3f723bf01235 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:31.099122 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd5d18-a52c-48f2-a191-3f723bf01235 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:31.133614 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 864/1731] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:31.232649 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:31.239093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eb4f97f2-70bd-41d6-9e3c-e8d9b5817b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af5557669984780a427691d752922a1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:31.241573 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eb4f97f2-70bd-41d6-9e3c-e8d9b5817b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af5557669984780a427691d752922a1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:31.260985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eb4f97f2-70bd-41d6-9e3c-e8d9b5817b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af5557669984780a427691d752922a1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:31.277890 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eb4f97f2-70bd-41d6-9e3c-e8d9b5817b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af5557669984780a427691d752922a1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:31.312301 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eb4f97f2-70bd-41d6-9e3c-e8d9b5817b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af5557669984780a427691d752922a1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:31.332759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 868/1732] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:06:30 2022] DELETE /identity/v3/users/9af5557669984780a427691d752922a1 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:31.337106 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbf77e-a2b6-40f0-bab5-ccd477b83419 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:31.338391 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbf77e-a2b6-40f0-bab5-ccd477b83419 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:31.338828 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbf77e-a2b6-40f0-bab5-ccd477b83419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:31.351598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:31.398857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f2cbf77e-a2b6-40f0-bab5-ccd477b83419 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:31.409290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f2cbf77e-a2b6-40f0-bab5-ccd477b83419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:31.569525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 865/1733] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:31.617046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f97f2-70bd-41d6-9e3c-e8d9b5817b6c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:31.618894 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-eb4f97f2-70bd-41d6-9e3c-e8d9b5817b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:31.752392 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f97f2-70bd-41d6-9e3c-e8d9b5817b6c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:31.765051 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeba802-2083-493d-98c2-c70430a404ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:31.766355 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeba802-2083-493d-98c2-c70430a404ff service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:31.767397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeba802-2083-493d-98c2-c70430a404ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:31.768023 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeba802-2083-493d-98c2-c70430a404ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:31.819132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeba802-2083-493d-98c2-c70430a404ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:31.820648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeba802-2083-493d-98c2-c70430a404ff service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:31.853287 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 869/1734] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:33.659461 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1affe6f-5ce6-48ca-8b17-922c5c8099b4 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:33.660424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1affe6f-5ce6-48ca-8b17-922c5c8099b4 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:33.660705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1affe6f-5ce6-48ca-8b17-922c5c8099b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:33.702348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a1affe6f-5ce6-48ca-8b17-922c5c8099b4 None None] MFA Rules not processed for user `d00634cdd3fb41e794d3af0b88ba7338`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:33.706344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a1affe6f-5ce6-48ca-8b17-922c5c8099b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:33.715285 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 866/1735] 10.222.0.73 () {58 vars in 1015 bytes} [Fri Sep 9 12:06:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:33.753416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeba802-2083-493d-98c2-c70430a404ff service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:33.755333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fbeba802-2083-493d-98c2-c70430a404ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:33.781944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeba802-2083-493d-98c2-c70430a404ff service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:33.793944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e6ec8-c229-4bd5-8644-36e22a3eff68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:33.794813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e6ec8-c229-4bd5-8644-36e22a3eff68 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:33.797716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e6ec8-c229-4bd5-8644-36e22a3eff68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:33.798221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e6ec8-c229-4bd5-8644-36e22a3eff68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:33.822432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e6ec8-c229-4bd5-8644-36e22a3eff68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:33.827548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e6ec8-c229-4bd5-8644-36e22a3eff68 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:33.851417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 870/1736] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:35.062804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1affe6f-5ce6-48ca-8b17-922c5c8099b4 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:35.065920 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a1affe6f-5ce6-48ca-8b17-922c5c8099b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:35.322589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1affe6f-5ce6-48ca-8b17-922c5c8099b4 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:35.344752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a6b20-6540-444d-a05e-865318e564ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:35.345799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a6b20-6540-444d-a05e-865318e564ec service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:35.346886 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a6b20-6540-444d-a05e-865318e564ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:35.347453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a6b20-6540-444d-a05e-865318e564ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:35.363895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-3b8a6b20-6540-444d-a05e-865318e564ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:35.522901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a6b20-6540-444d-a05e-865318e564ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:35.524480 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a6b20-6540-444d-a05e-865318e564ec service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:35.542348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 867/1737] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:35.989454 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e6ec8-c229-4bd5-8644-36e22a3eff68 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:35.992381 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-451e6ec8-c229-4bd5-8644-36e22a3eff68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:36.039371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e6ec8-c229-4bd5-8644-36e22a3eff68 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:36.054444 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1f17a-d070-4991-9431-01cc66f1561d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:36.055428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1f17a-d070-4991-9431-01cc66f1561d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:36.055884 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1f17a-d070-4991-9431-01cc66f1561d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:36.056383 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1f17a-d070-4991-9431-01cc66f1561d admin admin] PATH_INFO: `/v3/projects/414b3c8f1b29468684a54865d838effe` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:36.058729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1f17a-d070-4991-9431-01cc66f1561d admin admin] RBAC: Authorizing `identity:delete_project(project_id=414b3c8f1b29468684a54865d838effe)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:36.060910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1f17a-d070-4991-9431-01cc66f1561d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:36.188705 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1bc1f17a-d070-4991-9431-01cc66f1561d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '414b3c8f1b29468684a54865d838effe', 'request_id': 'req-1bc1f17a-d070-4991-9431-01cc66f1561d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:36.214436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1bc1f17a-d070-4991-9431-01cc66f1561d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '414b3c8f1b29468684a54865d838effe', 'request_id': 'req-1bc1f17a-d070-4991-9431-01cc66f1561d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:36.228571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1bc1f17a-d070-4991-9431-01cc66f1561d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '414b3c8f1b29468684a54865d838effe', 'request_id': 'req-1bc1f17a-d070-4991-9431-01cc66f1561d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:36.264976 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1bc1f17a-d070-4991-9431-01cc66f1561d admin admin] The token cache is being invalidate because project 414b3c8f1b29468684a54865d838effe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:06:36.265283 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1bc1f17a-d070-4991-9431-01cc66f1561d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:36.279183 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdcbb5-2a61-43fa-b917-db71ab9157eb None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:36.279677 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdcbb5-2a61-43fa-b917-db71ab9157eb None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:36.280048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdcbb5-2a61-43fa-b917-db71ab9157eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:36.281168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1bc1f17a-d070-4991-9431-01cc66f1561d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:36.286955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 871/1738] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:06:35 2022] DELETE /identity/v3/projects/414b3c8f1b29468684a54865d838effe => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:36.292326 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06854935-f3fb-4553-9de1-65aac99a460e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:36.292907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06854935-f3fb-4553-9de1-65aac99a460e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:36.293634 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06854935-f3fb-4553-9de1-65aac99a460e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:36.350363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-06854935-f3fb-4553-9de1-65aac99a460e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:36.351848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-06854935-f3fb-4553-9de1-65aac99a460e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:36.393242 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f1bdcbb5-2a61-43fa-b917-db71ab9157eb None None] MFA Rules not processed for user `16c99b5db9794dff83ea1b67fd238655`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:36.399407 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f1bdcbb5-2a61-43fa-b917-db71ab9157eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:36.506814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:36.663837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:36.684835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 872/1739] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:06:36.689557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e9614-bfb4-415a-8a01-0154a4dc11b5 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:36.692473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e9614-bfb4-415a-8a01-0154a4dc11b5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:36.693517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e9614-bfb4-415a-8a01-0154a4dc11b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:36.795936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-701e9614-bfb4-415a-8a01-0154a4dc11b5 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:36.803975 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-701e9614-bfb4-415a-8a01-0154a4dc11b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:36.804220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 868/1740] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 524 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:06:36.835312 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 873/1741] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:36.841454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdcbb5-2a61-43fa-b917-db71ab9157eb None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:36.867708 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdcbb5-2a61-43fa-b917-db71ab9157eb None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:36.882207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb576b-5201-4e86-abd0-4e260b756142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:36.882796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb576b-5201-4e86-abd0-4e260b756142 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:36.885154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb576b-5201-4e86-abd0-4e260b756142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:36.885758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb576b-5201-4e86-abd0-4e260b756142 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:36.886585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb576b-5201-4e86-abd0-4e260b756142 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:36.886838 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb576b-5201-4e86-abd0-4e260b756142 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:36.923858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb576b-5201-4e86-abd0-4e260b756142 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:36.959108 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 869/1742] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:06:36.964520 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e9614-bfb4-415a-8a01-0154a4dc11b5 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:36.989276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e9614-bfb4-415a-8a01-0154a4dc11b5 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:36.997461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f478d-7755-43de-9f5f-a31ccc95b4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:36.999435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f478d-7755-43de-9f5f-a31ccc95b4dc admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:36.999943 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f478d-7755-43de-9f5f-a31ccc95b4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:37.000436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f478d-7755-43de-9f5f-a31ccc95b4dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:37.001571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f478d-7755-43de-9f5f-a31ccc95b4dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:37.003849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f478d-7755-43de-9f5f-a31ccc95b4dc admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:37.082908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 874/1743] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:36 2022] POST /identity/v3/projects => generated 367 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:37.087778 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb576b-5201-4e86-abd0-4e260b756142 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:37.102576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb576b-5201-4e86-abd0-4e260b756142 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:37.181547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f478d-7755-43de-9f5f-a31ccc95b4dc admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:37.183879 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b00f478d-7755-43de-9f5f-a31ccc95b4dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:37.308037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10491b2-7b3b-45bf-a69f-9b2c56aba55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:37.312564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10491b2-7b3b-45bf-a69f-9b2c56aba55b admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:37.313480 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10491b2-7b3b-45bf-a69f-9b2c56aba55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:37.313953 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10491b2-7b3b-45bf-a69f-9b2c56aba55b admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:37.317512 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10491b2-7b3b-45bf-a69f-9b2c56aba55b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:37.327004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10491b2-7b3b-45bf-a69f-9b2c56aba55b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:37.394618 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 870/1744] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:37 2022] POST /identity/v3/users => generated 284 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:37.397940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10491b2-7b3b-45bf-a69f-9b2c56aba55b admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:37.455942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10491b2-7b3b-45bf-a69f-9b2c56aba55b admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:37.486933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3b798-a183-4a1d-8e1c-abcd1e3ff93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:37.488244 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3b798-a183-4a1d-8e1c-abcd1e3ff93a admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:37.490066 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3b798-a183-4a1d-8e1c-abcd1e3ff93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:37.490650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3b798-a183-4a1d-8e1c-abcd1e3ff93a admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:37.491422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3b798-a183-4a1d-8e1c-abcd1e3ff93a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:37.498688 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3b798-a183-4a1d-8e1c-abcd1e3ff93a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:37.518187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f478d-7755-43de-9f5f-a31ccc95b4dc admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:37.545450 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082f304-5d93-4315-b1e6-d0bff7b6a33f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:37.546412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 871/1745] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:37.547645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3b798-a183-4a1d-8e1c-abcd1e3ff93a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:37.548254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082f304-5d93-4315-b1e6-d0bff7b6a33f service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:37.549235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082f304-5d93-4315-b1e6-d0bff7b6a33f service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:37.551409 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082f304-5d93-4315-b1e6-d0bff7b6a33f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:37.608443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3b798-a183-4a1d-8e1c-abcd1e3ff93a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:37.665950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464b814-2f12-4549-a418-d9dcddac19d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:37.675759 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464b814-2f12-4549-a418-d9dcddac19d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:37.676107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464b814-2f12-4549-a418-d9dcddac19d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:37.678160 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464b814-2f12-4549-a418-d9dcddac19d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:37.679354 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464b814-2f12-4549-a418-d9dcddac19d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:37.688546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464b814-2f12-4549-a418-d9dcddac19d1 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:37.755658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 872/1746] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:37.758006 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464b814-2f12-4549-a418-d9dcddac19d1 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:37.766593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082f304-5d93-4315-b1e6-d0bff7b6a33f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:37.773579 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082f304-5d93-4315-b1e6-d0bff7b6a33f service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:37.792394 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464b814-2f12-4549-a418-d9dcddac19d1 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:37.799888 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 875/1747] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:06:37.804024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9856-3164-42c2-8ad8-b12984379957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:37.805585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9856-3164-42c2-8ad8-b12984379957 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:37.806662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9856-3164-42c2-8ad8-b12984379957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:37.807072 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9856-3164-42c2-8ad8-b12984379957 admin admin] PATH_INFO: `/v3/projects/a88dca3f1f934b2e812a1ffd3977b480/users/6ef4b0ce04da4e969efab2861fb6bf46/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:37.852412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe9856-3164-42c2-8ad8-b12984379957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a88dca3f1f934b2e812a1ffd3977b480, user_id=6ef4b0ce04da4e969efab2861fb6bf46, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:37.864909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe9856-3164-42c2-8ad8-b12984379957 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:37.898741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 873/1748] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:37 2022] PUT /identity/v3/projects/a88dca3f1f934b2e812a1ffd3977b480/users/6ef4b0ce04da4e969efab2861fb6bf46/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:37.912509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790047b-9eb0-44e6-aec0-bfefdf380718 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:37.912834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790047b-9eb0-44e6-aec0-bfefdf380718 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:37.913067 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790047b-9eb0-44e6-aec0-bfefdf380718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:38.039576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4790047b-9eb0-44e6-aec0-bfefdf380718 None None] MFA Rules not processed for user `6ef4b0ce04da4e969efab2861fb6bf46`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:38.040991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4790047b-9eb0-44e6-aec0-bfefdf380718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:38.213850 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:38.455263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 876/1749] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 543 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:38.516193 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9856-3164-42c2-8ad8-b12984379957 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:38.525720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-bffe9856-3164-42c2-8ad8-b12984379957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:38.732998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9856-3164-42c2-8ad8-b12984379957 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:38.750407 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f662ad1-f742-47dd-b8d0-4fd11ac90476 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:38.752759 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f662ad1-f742-47dd-b8d0-4fd11ac90476 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:38.753056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f662ad1-f742-47dd-b8d0-4fd11ac90476 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:38.753340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f662ad1-f742-47dd-b8d0-4fd11ac90476 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:38.781730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f662ad1-f742-47dd-b8d0-4fd11ac90476 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:38.783716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f662ad1-f742-47dd-b8d0-4fd11ac90476 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:38.829847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 874/1750] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:40.281854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790047b-9eb0-44e6-aec0-bfefdf380718 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:40.284459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4790047b-9eb0-44e6-aec0-bfefdf380718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:40.563826 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790047b-9eb0-44e6-aec0-bfefdf380718 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:40.580098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e40b40-b0b7-44d3-b186-61250f7d1a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:40.580793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e40b40-b0b7-44d3-b186-61250f7d1a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:40.581057 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e40b40-b0b7-44d3-b186-61250f7d1a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:40.581327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e40b40-b0b7-44d3-b186-61250f7d1a33 admin admin] PATH_INFO: `/v3/users/8ca25551d5704ec4b432c40ad4916a3a` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:40.583986 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e40b40-b0b7-44d3-b186-61250f7d1a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ca25551d5704ec4b432c40ad4916a3a)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:40.590761 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e40b40-b0b7-44d3-b186-61250f7d1a33 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:40.770238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f4e40b40-b0b7-44d3-b186-61250f7d1a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ca25551d5704ec4b432c40ad4916a3a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:40.771428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f4e40b40-b0b7-44d3-b186-61250f7d1a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ca25551d5704ec4b432c40ad4916a3a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:40.780186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f4e40b40-b0b7-44d3-b186-61250f7d1a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ca25551d5704ec4b432c40ad4916a3a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:40.790917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f4e40b40-b0b7-44d3-b186-61250f7d1a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ca25551d5704ec4b432c40ad4916a3a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:40.825036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f4e40b40-b0b7-44d3-b186-61250f7d1a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ca25551d5704ec4b432c40ad4916a3a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:40.840938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 877/1751] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:06:40 2022] DELETE /identity/v3/users/8ca25551d5704ec4b432c40ad4916a3a => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:40.845706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb8b3f-a82a-41cd-8f04-762aed29562f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:40.847198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb8b3f-a82a-41cd-8f04-762aed29562f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:40.847896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb8b3f-a82a-41cd-8f04-762aed29562f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:40.926594 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-8bcb8b3f-a82a-41cd-8f04-762aed29562f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:40.936359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-8bcb8b3f-a82a-41cd-8f04-762aed29562f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:40.982412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:41.220787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 875/1752] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:41.318691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c3910-32ae-47c6-875a-5aeb350ec72d None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:41.320637 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c3910-32ae-47c6-875a-5aeb350ec72d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:41.321164 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c3910-32ae-47c6-875a-5aeb350ec72d None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:41.323544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 878/1753] 10.222.0.62 () {54 vars in 921 bytes} [Fri Sep 9 12:06:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:41.450830 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3bb5b-8be7-405f-9b8b-da200c0d5c30 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:41.451514 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3bb5b-8be7-405f-9b8b-da200c0d5c30 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:41.452008 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3bb5b-8be7-405f-9b8b-da200c0d5c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:41.500772 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-32a3bb5b-8be7-405f-9b8b-da200c0d5c30 None None] MFA Rules not processed for user `1b30e80c421245b5a1ed5849b28ea90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:41.503190 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c3910-32ae-47c6-875a-5aeb350ec72d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:41.505565 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0c6c3910-32ae-47c6-875a-5aeb350ec72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:41.518262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-32a3bb5b-8be7-405f-9b8b-da200c0d5c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:41.831056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c3910-32ae-47c6-875a-5aeb350ec72d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:41.841893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584011ab-5b0c-46f6-a999-8dfc841fe064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:41.842771 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584011ab-5b0c-46f6-a999-8dfc841fe064 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:41.843384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584011ab-5b0c-46f6-a999-8dfc841fe064 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:41.843702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584011ab-5b0c-46f6-a999-8dfc841fe064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:41.862599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:41.871927 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584011ab-5b0c-46f6-a999-8dfc841fe064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:41.877252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584011ab-5b0c-46f6-a999-8dfc841fe064 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:41.909051 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 879/1754] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:06:41.964887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 876/1755] 10.222.0.62 () {58 vars in 1018 bytes} [Fri Sep 9 12:06:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 515 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:42.045460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584011ab-5b0c-46f6-a999-8dfc841fe064 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:42.045460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-584011ab-5b0c-46f6-a999-8dfc841fe064 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:42.069774 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584011ab-5b0c-46f6-a999-8dfc841fe064 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:42.086578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437811a-1284-4315-9499-5f8c1e2da6c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:42.088421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437811a-1284-4315-9499-5f8c1e2da6c0 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:42.088720 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437811a-1284-4315-9499-5f8c1e2da6c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:42.089050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437811a-1284-4315-9499-5f8c1e2da6c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:42.122702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437811a-1284-4315-9499-5f8c1e2da6c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:42.131566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437811a-1284-4315-9499-5f8c1e2da6c0 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:42.178102 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 880/1756] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:43.273930 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0ecf8-9c72-41be-b0d4-60661b259ea7 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:43.274609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0ecf8-9c72-41be-b0d4-60661b259ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:43.274705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0ecf8-9c72-41be-b0d4-60661b259ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:43.325306 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-0df0ecf8-9c72-41be-b0d4-60661b259ea7 None None] MFA Rules not processed for user `9676597a1b884138bf5435f3b6cd63c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:43.327237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0df0ecf8-9c72-41be-b0d4-60661b259ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:43.516756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 877/1757] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:43.540271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31847e0-862a-42c6-8f91-0f0e2e205225 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:43.540891 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31847e0-862a-42c6-8f91-0f0e2e205225 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:43.540891 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31847e0-862a-42c6-8f91-0f0e2e205225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:43.592853 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c31847e0-862a-42c6-8f91-0f0e2e205225 None None] MFA Rules not processed for user `bfc29d15a7344754aa53d55379e2751f`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:43.594654 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c31847e0-862a-42c6-8f91-0f0e2e205225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:43.709031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63c3c3-ea19-4c23-87db-5fdee421bbc6 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:43.709404 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63c3c3-ea19-4c23-87db-5fdee421bbc6 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:43.709851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63c3c3-ea19-4c23-87db-5fdee421bbc6 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:43.713351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 878/1758] 10.222.0.62 () {54 vars in 921 bytes} [Fri Sep 9 12:06:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:43.838342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52806004-a00f-48d0-a310-56168db4bfff None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:43.838652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52806004-a00f-48d0-a310-56168db4bfff None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:43.839229 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52806004-a00f-48d0-a310-56168db4bfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:43.844523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 881/1759] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:43.928540 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-52806004-a00f-48d0-a310-56168db4bfff None None] MFA Rules not processed for user `1b30e80c421245b5a1ed5849b28ea90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:43.929965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-52806004-a00f-48d0-a310-56168db4bfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:44.008552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 879/1760] 10.222.0.62 () {58 vars in 1018 bytes} [Fri Sep 9 12:06:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:06:45.518057 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31847e0-862a-42c6-8f91-0f0e2e205225 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:45.536481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31847e0-862a-42c6-8f91-0f0e2e205225 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:45.547534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308d7b6-8753-431d-93cf-ab10f82279a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:45.548683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308d7b6-8753-431d-93cf-ab10f82279a5 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:45.549114 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308d7b6-8753-431d-93cf-ab10f82279a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:45.549683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308d7b6-8753-431d-93cf-ab10f82279a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:45.563513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308d7b6-8753-431d-93cf-ab10f82279a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:45.567765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308d7b6-8753-431d-93cf-ab10f82279a5 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:45.596639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 882/1761] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:45.639639 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52806004-a00f-48d0-a310-56168db4bfff None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:45.641347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-52806004-a00f-48d0-a310-56168db4bfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:45.675000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52806004-a00f-48d0-a310-56168db4bfff None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:45.685068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3597137-9075-41bf-ba1a-37e23e59eb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:45.685895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3597137-9075-41bf-ba1a-37e23e59eb8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:45.686348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3597137-9075-41bf-ba1a-37e23e59eb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:45.686772 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3597137-9075-41bf-ba1a-37e23e59eb8a admin admin] PATH_INFO: `/v3/projects/ab4dbbb8fd204c1a84635df66e1b24f2` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:45.689615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3597137-9075-41bf-ba1a-37e23e59eb8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab4dbbb8fd204c1a84635df66e1b24f2)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:45.691741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3597137-9075-41bf-ba1a-37e23e59eb8a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:45.781873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:45.807153 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f3597137-9075-41bf-ba1a-37e23e59eb8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab4dbbb8fd204c1a84635df66e1b24f2', 'request_id': 'req-f3597137-9075-41bf-ba1a-37e23e59eb8a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:45.850082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f3597137-9075-41bf-ba1a-37e23e59eb8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab4dbbb8fd204c1a84635df66e1b24f2', 'request_id': 'req-f3597137-9075-41bf-ba1a-37e23e59eb8a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:45.860539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f3597137-9075-41bf-ba1a-37e23e59eb8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab4dbbb8fd204c1a84635df66e1b24f2', 'request_id': 'req-f3597137-9075-41bf-ba1a-37e23e59eb8a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:45.877617 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f3597137-9075-41bf-ba1a-37e23e59eb8a admin admin] The token cache is being invalidate because project ab4dbbb8fd204c1a84635df66e1b24f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:06:45.877917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f3597137-9075-41bf-ba1a-37e23e59eb8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:45.878877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f3597137-9075-41bf-ba1a-37e23e59eb8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:45.885864 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 880/1762] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:06:45 2022] DELETE /identity/v3/projects/ab4dbbb8fd204c1a84635df66e1b24f2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:45.886538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308d7b6-8753-431d-93cf-ab10f82279a5 service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:45.908609 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308d7b6-8753-431d-93cf-ab10f82279a5 service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:45.998242 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80f114-587d-4e37-a5e8-15fcc80b11aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:45.998902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80f114-587d-4e37-a5e8-15fcc80b11aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:45.999230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80f114-587d-4e37-a5e8-15fcc80b11aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:45.999506 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80f114-587d-4e37-a5e8-15fcc80b11aa admin admin] PATH_INFO: `/v3/users/61c303082a4845889916f0cfcbc57e37` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:46.001211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80f114-587d-4e37-a5e8-15fcc80b11aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=61c303082a4845889916f0cfcbc57e37)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:46.003476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80f114-587d-4e37-a5e8-15fcc80b11aa admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:46.070685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1c80f114-587d-4e37-a5e8-15fcc80b11aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61c303082a4845889916f0cfcbc57e37'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:46.071483 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1c80f114-587d-4e37-a5e8-15fcc80b11aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61c303082a4845889916f0cfcbc57e37'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:46.082083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1c80f114-587d-4e37-a5e8-15fcc80b11aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61c303082a4845889916f0cfcbc57e37'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:46.093812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1c80f114-587d-4e37-a5e8-15fcc80b11aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61c303082a4845889916f0cfcbc57e37'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:46.107344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:46.116216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1c80f114-587d-4e37-a5e8-15fcc80b11aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61c303082a4845889916f0cfcbc57e37'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:46.134528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 883/1763] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:06:45 2022] DELETE /identity/v3/users/61c303082a4845889916f0cfcbc57e37 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:06:46.879263 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418cc3e-32e4-4e16-905f-285bf5d62fe8 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:46.879823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418cc3e-32e4-4e16-905f-285bf5d62fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:46.880081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418cc3e-32e4-4e16-905f-285bf5d62fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:46.952303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a418cc3e-32e4-4e16-905f-285bf5d62fe8 None None] MFA Rules not processed for user `16c99b5db9794dff83ea1b67fd238655`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:46.965725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a418cc3e-32e4-4e16-905f-285bf5d62fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:47.101623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 881/1764] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:47.187070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80f114-587d-4e37-a5e8-15fcc80b11aa admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:47.190883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1c80f114-587d-4e37-a5e8-15fcc80b11aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:47.381665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80f114-587d-4e37-a5e8-15fcc80b11aa admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:47.388575 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6153097-cc96-4465-a575-decd0603e58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:47.389227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6153097-cc96-4465-a575-decd0603e58b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:47.389502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6153097-cc96-4465-a575-decd0603e58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:47.389830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6153097-cc96-4465-a575-decd0603e58b admin admin] PATH_INFO: `/v3/projects/26a6bdd442d648a29637e52798ba11ff` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:47.391140 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6153097-cc96-4465-a575-decd0603e58b admin admin] RBAC: Authorizing `identity:delete_project(project_id=26a6bdd442d648a29637e52798ba11ff)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:47.393099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6153097-cc96-4465-a575-decd0603e58b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:47.456486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b6153097-cc96-4465-a575-decd0603e58b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26a6bdd442d648a29637e52798ba11ff', 'request_id': 'req-b6153097-cc96-4465-a575-decd0603e58b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:47.477077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b6153097-cc96-4465-a575-decd0603e58b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26a6bdd442d648a29637e52798ba11ff', 'request_id': 'req-b6153097-cc96-4465-a575-decd0603e58b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:47.489321 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b6153097-cc96-4465-a575-decd0603e58b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26a6bdd442d648a29637e52798ba11ff', 'request_id': 'req-b6153097-cc96-4465-a575-decd0603e58b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:47.517434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b6153097-cc96-4465-a575-decd0603e58b admin admin] The token cache is being invalidate because project 26a6bdd442d648a29637e52798ba11ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:06:47.517817 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b6153097-cc96-4465-a575-decd0603e58b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:47.518598 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b6153097-cc96-4465-a575-decd0603e58b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:47.523357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 884/1765] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:06:47 2022] DELETE /identity/v3/projects/26a6bdd442d648a29637e52798ba11ff => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:47.525640 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418cc3e-32e4-4e16-905f-285bf5d62fe8 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:47.544968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418cc3e-32e4-4e16-905f-285bf5d62fe8 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:47.670630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509afa20-9faa-47ee-85ed-35a646a80d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:47.675441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509afa20-9faa-47ee-85ed-35a646a80d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:47.675716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509afa20-9faa-47ee-85ed-35a646a80d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:47.676147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509afa20-9faa-47ee-85ed-35a646a80d13 admin admin] PATH_INFO: `/v3/users/ee00b721f06f4aacaaf83a6a96b67c4d` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:47.678311 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509afa20-9faa-47ee-85ed-35a646a80d13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee00b721f06f4aacaaf83a6a96b67c4d)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:47.680453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509afa20-9faa-47ee-85ed-35a646a80d13 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:47.746994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-509afa20-9faa-47ee-85ed-35a646a80d13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee00b721f06f4aacaaf83a6a96b67c4d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:47.756582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-509afa20-9faa-47ee-85ed-35a646a80d13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee00b721f06f4aacaaf83a6a96b67c4d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:47.771995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-509afa20-9faa-47ee-85ed-35a646a80d13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee00b721f06f4aacaaf83a6a96b67c4d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:47.774074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:47.780791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-509afa20-9faa-47ee-85ed-35a646a80d13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee00b721f06f4aacaaf83a6a96b67c4d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:47.782399 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-509afa20-9faa-47ee-85ed-35a646a80d13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee00b721f06f4aacaaf83a6a96b67c4d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:47.798421 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 882/1766] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:06:47 2022] DELETE /identity/v3/users/ee00b721f06f4aacaaf83a6a96b67c4d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:48.152903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6153097-cc96-4465-a575-decd0603e58b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:48.152903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b6153097-cc96-4465-a575-decd0603e58b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:48.182719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6153097-cc96-4465-a575-decd0603e58b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:48.296792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6b56c-edb7-4289-b30a-6f57be8fea8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:48.298513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6b56c-edb7-4289-b30a-6f57be8fea8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:48.298933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6b56c-edb7-4289-b30a-6f57be8fea8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:48.299472 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6b56c-edb7-4289-b30a-6f57be8fea8c admin admin] PATH_INFO: `/v3/projects/dfe5946c69b7499fbdfb8991d65729a2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:48.301636 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6b56c-edb7-4289-b30a-6f57be8fea8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfe5946c69b7499fbdfb8991d65729a2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:48.305205 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6b56c-edb7-4289-b30a-6f57be8fea8c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:48.386976 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:48.423838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3de6b56c-edb7-4289-b30a-6f57be8fea8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfe5946c69b7499fbdfb8991d65729a2', 'request_id': 'req-3de6b56c-edb7-4289-b30a-6f57be8fea8c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:48.443109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3de6b56c-edb7-4289-b30a-6f57be8fea8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfe5946c69b7499fbdfb8991d65729a2', 'request_id': 'req-3de6b56c-edb7-4289-b30a-6f57be8fea8c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:48.452559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3de6b56c-edb7-4289-b30a-6f57be8fea8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfe5946c69b7499fbdfb8991d65729a2', 'request_id': 'req-3de6b56c-edb7-4289-b30a-6f57be8fea8c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:48.496031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3de6b56c-edb7-4289-b30a-6f57be8fea8c admin admin] The token cache is being invalidate because project dfe5946c69b7499fbdfb8991d65729a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:06:48.496334 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3de6b56c-edb7-4289-b30a-6f57be8fea8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:48.497159 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3de6b56c-edb7-4289-b30a-6f57be8fea8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:48.498938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 885/1767] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:06:48 2022] DELETE /identity/v3/projects/dfe5946c69b7499fbdfb8991d65729a2 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:48.505297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849b85a-c704-407c-bcae-664605759ccc None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:48.505763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849b85a-c704-407c-bcae-664605759ccc None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:48.507821 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849b85a-c704-407c-bcae-664605759ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:48.540780 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d849b85a-c704-407c-bcae-664605759ccc None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:48.542396 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d849b85a-c704-407c-bcae-664605759ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:48.650282 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 883/1768] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:48.654313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120991e-10f6-47b7-a182-2af805f8ab30 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:48.654620 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120991e-10f6-47b7-a182-2af805f8ab30 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:48.654874 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120991e-10f6-47b7-a182-2af805f8ab30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:48.678871 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7120991e-10f6-47b7-a182-2af805f8ab30 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:48.680212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7120991e-10f6-47b7-a182-2af805f8ab30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:48.694366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 886/1769] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:06:48.695476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849b85a-c704-407c-bcae-664605759ccc None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:48.707329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849b85a-c704-407c-bcae-664605759ccc None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:48.713575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3868804-d89c-49c6-8d8b-3dde6ad32ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:48.714898 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3868804-d89c-49c6-8d8b-3dde6ad32ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:48.715436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3868804-d89c-49c6-8d8b-3dde6ad32ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:48.715866 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3868804-d89c-49c6-8d8b-3dde6ad32ec0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:48.716658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3868804-d89c-49c6-8d8b-3dde6ad32ec0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:48.717130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3868804-d89c-49c6-8d8b-3dde6ad32ec0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:48.718910 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3868804-d89c-49c6-8d8b-3dde6ad32ec0 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:48.732541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 884/1770] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:06:48.736874 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120991e-10f6-47b7-a182-2af805f8ab30 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:48.746931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120991e-10f6-47b7-a182-2af805f8ab30 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:48.751695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47dfa9-b683-4562-939b-704b9f516dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:48.752306 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47dfa9-b683-4562-939b-704b9f516dcd admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:48.752581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47dfa9-b683-4562-939b-704b9f516dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:48.752852 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47dfa9-b683-4562-939b-704b9f516dcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:48.753488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47dfa9-b683-4562-939b-704b9f516dcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:48.762099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47dfa9-b683-4562-939b-704b9f516dcd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:48.795961 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 887/1771] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:48 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:48.797191 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3868804-d89c-49c6-8d8b-3dde6ad32ec0 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:48.808825 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3868804-d89c-49c6-8d8b-3dde6ad32ec0 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:48.875166 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29c137-bb07-4098-8689-516b7e696c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:48.875775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29c137-bb07-4098-8689-516b7e696c2d admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:48.876061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29c137-bb07-4098-8689-516b7e696c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:48.876340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29c137-bb07-4098-8689-516b7e696c2d admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:48.877052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29c137-bb07-4098-8689-516b7e696c2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:48.879043 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29c137-bb07-4098-8689-516b7e696c2d admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:48.904368 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 885/1772] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:48 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:48.908511 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47dfa9-b683-4562-939b-704b9f516dcd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:48.919569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47dfa9-b683-4562-939b-704b9f516dcd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:48.924496 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0dfc2-0386-458a-8480-f0a6512500e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:48.925209 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0dfc2-0386-458a-8480-f0a6512500e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:48.925487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0dfc2-0386-458a-8480-f0a6512500e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:48.925755 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0dfc2-0386-458a-8480-f0a6512500e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:48.926466 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0dfc2-0386-458a-8480-f0a6512500e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:48.928037 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0dfc2-0386-458a-8480-f0a6512500e8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:48.938747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 888/1773] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:48.943009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29c137-bb07-4098-8689-516b7e696c2d admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:48.953105 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29c137-bb07-4098-8689-516b7e696c2d admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:48.958042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb55f69-e215-4d93-b13a-852221e9cdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:48.958735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb55f69-e215-4d93-b13a-852221e9cdd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:48.959078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb55f69-e215-4d93-b13a-852221e9cdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:48.959408 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb55f69-e215-4d93-b13a-852221e9cdd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:48.960143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb55f69-e215-4d93-b13a-852221e9cdd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:48.961788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb55f69-e215-4d93-b13a-852221e9cdd5 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:48.973613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 886/1774] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:06:48.978728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0dfc2-0386-458a-8480-f0a6512500e8 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:48.989371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0dfc2-0386-458a-8480-f0a6512500e8 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:48.994474 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9ce10-5a00-408a-b8c2-b9ce07a7761a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:48.995395 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9ce10-5a00-408a-b8c2-b9ce07a7761a admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:48.995680 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9ce10-5a00-408a-b8c2-b9ce07a7761a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:48.995948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9ce10-5a00-408a-b8c2-b9ce07a7761a admin admin] PATH_INFO: `/v3/projects/7ec498b7331f47c6949e294e3d4545c7/users/95018052070947089e10f51e459bfd13/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:49.014362 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9ce10-5a00-408a-b8c2-b9ce07a7761a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec498b7331f47c6949e294e3d4545c7, user_id=95018052070947089e10f51e459bfd13, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:49.016989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9ce10-5a00-408a-b8c2-b9ce07a7761a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:49.031353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 889/1775] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:48 2022] PUT /identity/v3/projects/7ec498b7331f47c6949e294e3d4545c7/users/95018052070947089e10f51e459bfd13/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:06:49.044358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fda15-24dd-4c14-b88b-68a1de3456fa None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:49.044645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fda15-24dd-4c14-b88b-68a1de3456fa None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:49.044933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fda15-24dd-4c14-b88b-68a1de3456fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:49.075533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-142fda15-24dd-4c14-b88b-68a1de3456fa None None] MFA Rules not processed for user `95018052070947089e10f51e459bfd13`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:49.076684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-142fda15-24dd-4c14-b88b-68a1de3456fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:49.125921 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:49.148042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 887/1776] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:49.153033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9ce10-5a00-408a-b8c2-b9ce07a7761a admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:49.213594 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9ce10-5a00-408a-b8c2-b9ce07a7761a admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:49.218290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4039bc-6d71-4fa4-8dac-6a80c1e1c67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:49.218901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4039bc-6d71-4fa4-8dac-6a80c1e1c67d admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:49.219181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4039bc-6d71-4fa4-8dac-6a80c1e1c67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:49.219484 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4039bc-6d71-4fa4-8dac-6a80c1e1c67d admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:49.220144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4039bc-6d71-4fa4-8dac-6a80c1e1c67d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:49.222042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4039bc-6d71-4fa4-8dac-6a80c1e1c67d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:49.249344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 890/1777] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:49 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:49.253616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fda15-24dd-4c14-b88b-68a1de3456fa None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:49.264954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fda15-24dd-4c14-b88b-68a1de3456fa None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:49.333493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd95ac-468f-45f6-80b8-43a062d3c446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:49.334131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd95ac-468f-45f6-80b8-43a062d3c446 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:49.334422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd95ac-468f-45f6-80b8-43a062d3c446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:49.334692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd95ac-468f-45f6-80b8-43a062d3c446 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:49.335338 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd95ac-468f-45f6-80b8-43a062d3c446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:49.337930 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd95ac-468f-45f6-80b8-43a062d3c446 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:49.374245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 888/1778] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:49 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:49.374361 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4039bc-6d71-4fa4-8dac-6a80c1e1c67d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:49.376068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4039bc-6d71-4fa4-8dac-6a80c1e1c67d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:49.380949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc159b6-d59d-412e-b021-227cf7564988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:49.381561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc159b6-d59d-412e-b021-227cf7564988 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:49.381836 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc159b6-d59d-412e-b021-227cf7564988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:49.382191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc159b6-d59d-412e-b021-227cf7564988 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:49.382870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc159b6-d59d-412e-b021-227cf7564988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:49.384972 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc159b6-d59d-412e-b021-227cf7564988 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:49.395893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 891/1779] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:06:49.400632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd95ac-468f-45f6-80b8-43a062d3c446 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:49.410682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd95ac-468f-45f6-80b8-43a062d3c446 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:49.415307 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c540319-9e83-459f-af55-bc9939c79c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:49.415954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c540319-9e83-459f-af55-bc9939c79c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:49.416236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c540319-9e83-459f-af55-bc9939c79c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:49.416508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c540319-9e83-459f-af55-bc9939c79c1b admin admin] PATH_INFO: `/v3/projects/dbedfb4751604b11bc0cada6c32b9ffb/users/61a2f92df64d46aaa5ab1614de65293e/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:49.431313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c540319-9e83-459f-af55-bc9939c79c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbedfb4751604b11bc0cada6c32b9ffb, user_id=61a2f92df64d46aaa5ab1614de65293e, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:49.434038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c540319-9e83-459f-af55-bc9939c79c1b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:49.447331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 889/1780] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:49 2022] PUT /identity/v3/projects/dbedfb4751604b11bc0cada6c32b9ffb/users/61a2f92df64d46aaa5ab1614de65293e/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:06:49.451342 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc159b6-d59d-412e-b021-227cf7564988 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:49.461477 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc159b6-d59d-412e-b021-227cf7564988 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:49.481118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:49.528035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49915108-ce23-422d-82e1-ac4ba328bab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:49.528937 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49915108-ce23-422d-82e1-ac4ba328bab4 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:49.529376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49915108-ce23-422d-82e1-ac4ba328bab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:49.529791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49915108-ce23-422d-82e1-ac4ba328bab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:49.530678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49915108-ce23-422d-82e1-ac4ba328bab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:49.532425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49915108-ce23-422d-82e1-ac4ba328bab4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:49.547424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 892/1781] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:49.551784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c540319-9e83-459f-af55-bc9939c79c1b admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:49.565110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c540319-9e83-459f-af55-bc9939c79c1b admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:49.571116 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154e1c6-bbb8-4f7a-a5dc-721c353b4ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:49.571919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154e1c6-bbb8-4f7a-a5dc-721c353b4ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:49.572310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154e1c6-bbb8-4f7a-a5dc-721c353b4ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:49.573006 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154e1c6-bbb8-4f7a-a5dc-721c353b4ed7 admin admin] PATH_INFO: `/v3/projects/dbedfb4751604b11bc0cada6c32b9ffb/users/61a2f92df64d46aaa5ab1614de65293e/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:49.574987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154e1c6-bbb8-4f7a-a5dc-721c353b4ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbedfb4751604b11bc0cada6c32b9ffb, user_id=61a2f92df64d46aaa5ab1614de65293e, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:49.577691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154e1c6-bbb8-4f7a-a5dc-721c353b4ed7 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:49.593707 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 890/1782] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:49 2022] PUT /identity/v3/projects/dbedfb4751604b11bc0cada6c32b9ffb/users/61a2f92df64d46aaa5ab1614de65293e/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:49.604429 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3264893-8a20-4904-9e45-0b6bd162dad6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:49.604717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3264893-8a20-4904-9e45-0b6bd162dad6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:49.604993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3264893-8a20-4904-9e45-0b6bd162dad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:49.635395 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e3264893-8a20-4904-9e45-0b6bd162dad6 None None] MFA Rules not processed for user `61a2f92df64d46aaa5ab1614de65293e`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:49.636608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e3264893-8a20-4904-9e45-0b6bd162dad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:49.711047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 893/1783] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:49.721965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154e1c6-bbb8-4f7a-a5dc-721c353b4ed7 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:49.723709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d154e1c6-bbb8-4f7a-a5dc-721c353b4ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:49.781811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154e1c6-bbb8-4f7a-a5dc-721c353b4ed7 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:49.785510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b6adf-8289-4147-b80d-0b76fcf55a0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:49.786172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b6adf-8289-4147-b80d-0b76fcf55a0b service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:49.786454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b6adf-8289-4147-b80d-0b76fcf55a0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:49.786726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b6adf-8289-4147-b80d-0b76fcf55a0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:49.796688 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b6adf-8289-4147-b80d-0b76fcf55a0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:49.798186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b6adf-8289-4147-b80d-0b76fcf55a0b service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:49.808300 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 891/1784] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:49.950966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3264893-8a20-4904-9e45-0b6bd162dad6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:50.015136 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3264893-8a20-4904-9e45-0b6bd162dad6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:50.019682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e0754-4410-4732-9cb6-570e1f7774cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:50.020323 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e0754-4410-4732-9cb6-570e1f7774cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:50.020601 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e0754-4410-4732-9cb6-570e1f7774cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:50.020872 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e0754-4410-4732-9cb6-570e1f7774cc admin admin] PATH_INFO: `/v3/users/95018052070947089e10f51e459bfd13` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:50.021993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e0754-4410-4732-9cb6-570e1f7774cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=95018052070947089e10f51e459bfd13)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:50.023923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e0754-4410-4732-9cb6-570e1f7774cc admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:50.071577 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:50.073784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-865e0754-4410-4732-9cb6-570e1f7774cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95018052070947089e10f51e459bfd13'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:50.074562 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-865e0754-4410-4732-9cb6-570e1f7774cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95018052070947089e10f51e459bfd13'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:50.083545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-865e0754-4410-4732-9cb6-570e1f7774cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95018052070947089e10f51e459bfd13'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:50.090853 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-865e0754-4410-4732-9cb6-570e1f7774cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95018052070947089e10f51e459bfd13'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:50.103665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-865e0754-4410-4732-9cb6-570e1f7774cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95018052070947089e10f51e459bfd13'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:50.109603 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 894/1785] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:06:49 2022] DELETE /identity/v3/users/95018052070947089e10f51e459bfd13 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:06:50.114632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3991639-2e0b-4023-91c4-c9710dcf2c06 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:50.114982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3991639-2e0b-4023-91c4-c9710dcf2c06 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:50.115324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3991639-2e0b-4023-91c4-c9710dcf2c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:50.140563 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d3991639-2e0b-4023-91c4-c9710dcf2c06 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:50.141765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d3991639-2e0b-4023-91c4-c9710dcf2c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:50.181288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:50.206986 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 892/1786] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:50.217176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e0754-4410-4732-9cb6-570e1f7774cc admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:50.218712 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-865e0754-4410-4732-9cb6-570e1f7774cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:50.280506 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e0754-4410-4732-9cb6-570e1f7774cc admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:50.284176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930651e2-ed94-4d3b-97c4-7b147e4e718b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:50.284784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930651e2-ed94-4d3b-97c4-7b147e4e718b service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:50.285063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930651e2-ed94-4d3b-97c4-7b147e4e718b service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:50.285341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930651e2-ed94-4d3b-97c4-7b147e4e718b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:50.294489 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930651e2-ed94-4d3b-97c4-7b147e4e718b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:50.295906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930651e2-ed94-4d3b-97c4-7b147e4e718b service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:50.305527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 895/1787] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:50.587557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3991639-2e0b-4023-91c4-c9710dcf2c06 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:50.589137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d3991639-2e0b-4023-91c4-c9710dcf2c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:50.607540 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3991639-2e0b-4023-91c4-c9710dcf2c06 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:50.612715 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1137d2-6766-4315-9db4-5f35c592643e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:50.621903 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1137d2-6766-4315-9db4-5f35c592643e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:50.622214 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1137d2-6766-4315-9db4-5f35c592643e admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:50.622488 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1137d2-6766-4315-9db4-5f35c592643e admin admin] PATH_INFO: `/v3/projects/7ec498b7331f47c6949e294e3d4545c7` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:50.623798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1137d2-6766-4315-9db4-5f35c592643e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec498b7331f47c6949e294e3d4545c7)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:50.625719 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1137d2-6766-4315-9db4-5f35c592643e admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:50.675903 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ce1137d2-6766-4315-9db4-5f35c592643e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec498b7331f47c6949e294e3d4545c7', 'request_id': 'req-ce1137d2-6766-4315-9db4-5f35c592643e'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:50.688692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ce1137d2-6766-4315-9db4-5f35c592643e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec498b7331f47c6949e294e3d4545c7', 'request_id': 'req-ce1137d2-6766-4315-9db4-5f35c592643e'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:50.696597 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ce1137d2-6766-4315-9db4-5f35c592643e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec498b7331f47c6949e294e3d4545c7', 'request_id': 'req-ce1137d2-6766-4315-9db4-5f35c592643e'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:50.708346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ce1137d2-6766-4315-9db4-5f35c592643e admin admin] The token cache is being invalidate because project 7ec498b7331f47c6949e294e3d4545c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:06:50.708629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ce1137d2-6766-4315-9db4-5f35c592643e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:50.709279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ce1137d2-6766-4315-9db4-5f35c592643e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:50.711304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 893/1788] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:06:50 2022] DELETE /identity/v3/projects/7ec498b7331f47c6949e294e3d4545c7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:50.715343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930651e2-ed94-4d3b-97c4-7b147e4e718b service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:50.725391 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930651e2-ed94-4d3b-97c4-7b147e4e718b service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:50.784890 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e6ee7-814c-4767-8a3f-05a9b67729fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:50.785530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e6ee7-814c-4767-8a3f-05a9b67729fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:50.785808 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e6ee7-814c-4767-8a3f-05a9b67729fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:50.786081 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e6ee7-814c-4767-8a3f-05a9b67729fb admin admin] PATH_INFO: `/v3/users/61a2f92df64d46aaa5ab1614de65293e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:50.787251 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e6ee7-814c-4767-8a3f-05a9b67729fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a2f92df64d46aaa5ab1614de65293e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:50.789172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e6ee7-814c-4767-8a3f-05a9b67729fb admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:50.839523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1c1e6ee7-814c-4767-8a3f-05a9b67729fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a2f92df64d46aaa5ab1614de65293e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:50.839523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1c1e6ee7-814c-4767-8a3f-05a9b67729fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a2f92df64d46aaa5ab1614de65293e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:50.847070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1c1e6ee7-814c-4767-8a3f-05a9b67729fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a2f92df64d46aaa5ab1614de65293e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:50.856156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1c1e6ee7-814c-4767-8a3f-05a9b67729fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a2f92df64d46aaa5ab1614de65293e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:50.861702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:50.872356 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1c1e6ee7-814c-4767-8a3f-05a9b67729fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a2f92df64d46aaa5ab1614de65293e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:50.881562 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 896/1789] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:06:50 2022] DELETE /identity/v3/users/61a2f92df64d46aaa5ab1614de65293e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:51.163095 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1137d2-6766-4315-9db4-5f35c592643e admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:51.164524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ce1137d2-6766-4315-9db4-5f35c592643e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:51.195812 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1137d2-6766-4315-9db4-5f35c592643e admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:51.259853 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a86ecf-e7f1-4a54-8e2e-d9dc789eaf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:51.260487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a86ecf-e7f1-4a54-8e2e-d9dc789eaf72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:51.260760 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a86ecf-e7f1-4a54-8e2e-d9dc789eaf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:51.261031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a86ecf-e7f1-4a54-8e2e-d9dc789eaf72 admin admin] PATH_INFO: `/v3/projects/dbedfb4751604b11bc0cada6c32b9ffb` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:51.262186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a86ecf-e7f1-4a54-8e2e-d9dc789eaf72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbedfb4751604b11bc0cada6c32b9ffb)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:51.264100 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a86ecf-e7f1-4a54-8e2e-d9dc789eaf72 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:51.328190 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-a3a86ecf-e7f1-4a54-8e2e-d9dc789eaf72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbedfb4751604b11bc0cada6c32b9ffb', 'request_id': 'req-a3a86ecf-e7f1-4a54-8e2e-d9dc789eaf72'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:51.336729 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-a3a86ecf-e7f1-4a54-8e2e-d9dc789eaf72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbedfb4751604b11bc0cada6c32b9ffb', 'request_id': 'req-a3a86ecf-e7f1-4a54-8e2e-d9dc789eaf72'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:51.348837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-a3a86ecf-e7f1-4a54-8e2e-d9dc789eaf72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbedfb4751604b11bc0cada6c32b9ffb', 'request_id': 'req-a3a86ecf-e7f1-4a54-8e2e-d9dc789eaf72'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:51.351890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:51.365401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-a3a86ecf-e7f1-4a54-8e2e-d9dc789eaf72 admin admin] The token cache is being invalidate because project dbedfb4751604b11bc0cada6c32b9ffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:06:51.365698 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-a3a86ecf-e7f1-4a54-8e2e-d9dc789eaf72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:51.366483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-a3a86ecf-e7f1-4a54-8e2e-d9dc789eaf72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:51.370168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 894/1790] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:06:51 2022] DELETE /identity/v3/projects/dbedfb4751604b11bc0cada6c32b9ffb => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:51.373678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9f016-fe97-47ad-b87a-bffb22d09f73 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:51.374070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9f016-fe97-47ad-b87a-bffb22d09f73 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:51.374439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9f016-fe97-47ad-b87a-bffb22d09f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:51.396727 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-73c9f016-fe97-47ad-b87a-bffb22d09f73 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:51.398251 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-73c9f016-fe97-47ad-b87a-bffb22d09f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:51.459937 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 897/1791] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:06:51.468783 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8aff31-657d-47b9-9fc6-981c07da525e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:51.469064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8aff31-657d-47b9-9fc6-981c07da525e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:51.469334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8aff31-657d-47b9-9fc6-981c07da525e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:51.490363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-bc8aff31-657d-47b9-9fc6-981c07da525e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:51.491559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-bc8aff31-657d-47b9-9fc6-981c07da525e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:51.500532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 895/1792] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:51.505113 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9f016-fe97-47ad-b87a-bffb22d09f73 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:51.515214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9f016-fe97-47ad-b87a-bffb22d09f73 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:51.520127 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09516f-bac5-4a2f-aa47-1549e26b3ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:51.520737 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09516f-bac5-4a2f-aa47-1549e26b3ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:51.521041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09516f-bac5-4a2f-aa47-1549e26b3ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:51.521316 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09516f-bac5-4a2f-aa47-1549e26b3ea4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:51.521967 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09516f-bac5-4a2f-aa47-1549e26b3ea4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:51.522327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09516f-bac5-4a2f-aa47-1549e26b3ea4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:51.523937 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09516f-bac5-4a2f-aa47-1549e26b3ea4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:51.539291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 898/1793] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:06:51.541474 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8aff31-657d-47b9-9fc6-981c07da525e None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:51.550839 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8aff31-657d-47b9-9fc6-981c07da525e None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:51.556168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04901c-bc36-4628-a196-77b77a6abe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:51.556827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04901c-bc36-4628-a196-77b77a6abe1e admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:51.557106 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04901c-bc36-4628-a196-77b77a6abe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:51.557376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04901c-bc36-4628-a196-77b77a6abe1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:51.558075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04901c-bc36-4628-a196-77b77a6abe1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:51.560122 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04901c-bc36-4628-a196-77b77a6abe1e admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:51.593364 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 896/1794] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:51 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:51.595038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09516f-bac5-4a2f-aa47-1549e26b3ea4 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:51.606574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09516f-bac5-4a2f-aa47-1549e26b3ea4 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:51.671039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be441c23-4c5b-4753-b72b-cc0c9147d416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:51.671655 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be441c23-4c5b-4753-b72b-cc0c9147d416 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:51.671934 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be441c23-4c5b-4753-b72b-cc0c9147d416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:51.672204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be441c23-4c5b-4753-b72b-cc0c9147d416 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:51.672836 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be441c23-4c5b-4753-b72b-cc0c9147d416 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:51.674830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be441c23-4c5b-4753-b72b-cc0c9147d416 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:51.702787 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 899/1795] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:51 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:51.704501 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04901c-bc36-4628-a196-77b77a6abe1e admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:51.716125 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04901c-bc36-4628-a196-77b77a6abe1e admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:51.721074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee968c-ed62-475c-85b0-ec0f963e3030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:51.721711 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee968c-ed62-475c-85b0-ec0f963e3030 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:51.721995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee968c-ed62-475c-85b0-ec0f963e3030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:51.722307 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee968c-ed62-475c-85b0-ec0f963e3030 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:51.722982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee968c-ed62-475c-85b0-ec0f963e3030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:51.724818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee968c-ed62-475c-85b0-ec0f963e3030 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:51.738962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 897/1796] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:51.742225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be441c23-4c5b-4753-b72b-cc0c9147d416 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:51.752729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be441c23-4c5b-4753-b72b-cc0c9147d416 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:51.757703 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7fafe6-4697-4e27-bf8a-f8efe7f21465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:51.758337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fafe6-4697-4e27-bf8a-f8efe7f21465 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:51.758610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fafe6-4697-4e27-bf8a-f8efe7f21465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:51.759416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fafe6-4697-4e27-bf8a-f8efe7f21465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:51.760103 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7fafe6-4697-4e27-bf8a-f8efe7f21465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:51.761664 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7fafe6-4697-4e27-bf8a-f8efe7f21465 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:51.771987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 900/1797] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:51.776890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee968c-ed62-475c-85b0-ec0f963e3030 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:51.786954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee968c-ed62-475c-85b0-ec0f963e3030 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:51.792522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b435e26-70b5-489f-a56b-f68fec2da29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:51.793244 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b435e26-70b5-489f-a56b-f68fec2da29b admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:51.793593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b435e26-70b5-489f-a56b-f68fec2da29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:51.793936 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b435e26-70b5-489f-a56b-f68fec2da29b admin admin] PATH_INFO: `/v3/projects/1bab48e5f3f34d179411e2e8ac66a982/users/495cf187b5114f19a83fa53f11a7b1b1/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:51.810215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b435e26-70b5-489f-a56b-f68fec2da29b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bab48e5f3f34d179411e2e8ac66a982, user_id=495cf187b5114f19a83fa53f11a7b1b1, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:51.812908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b435e26-70b5-489f-a56b-f68fec2da29b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:51.828275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 898/1798] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:51 2022] PUT /identity/v3/projects/1bab48e5f3f34d179411e2e8ac66a982/users/495cf187b5114f19a83fa53f11a7b1b1/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:51.838642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5efa9b-54ab-4526-914c-89a6f2b1e195 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:51.838949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5efa9b-54ab-4526-914c-89a6f2b1e195 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:51.839225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5efa9b-54ab-4526-914c-89a6f2b1e195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:51.867674 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ba5efa9b-54ab-4526-914c-89a6f2b1e195 None None] MFA Rules not processed for user `495cf187b5114f19a83fa53f11a7b1b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:51.869403 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ba5efa9b-54ab-4526-914c-89a6f2b1e195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:51.912814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:51.943356 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 901/1799] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:51.948649 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b435e26-70b5-489f-a56b-f68fec2da29b admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:52.042030 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b435e26-70b5-489f-a56b-f68fec2da29b admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:52.046786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f001176-79c9-4dc8-b12a-67fc67f58735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:52.047401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f001176-79c9-4dc8-b12a-67fc67f58735 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:52.047679 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f001176-79c9-4dc8-b12a-67fc67f58735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:52.047953 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f001176-79c9-4dc8-b12a-67fc67f58735 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:52.048600 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f001176-79c9-4dc8-b12a-67fc67f58735 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:52.050552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f001176-79c9-4dc8-b12a-67fc67f58735 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:52.083216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 899/1800] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:51 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:52.084286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5efa9b-54ab-4526-914c-89a6f2b1e195 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:52.095093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5efa9b-54ab-4526-914c-89a6f2b1e195 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:52.150359 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013084d3-28e1-48d8-9149-43a3cd34ed67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:52.150963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013084d3-28e1-48d8-9149-43a3cd34ed67 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:52.151236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013084d3-28e1-48d8-9149-43a3cd34ed67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:52.151501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013084d3-28e1-48d8-9149-43a3cd34ed67 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:52.152129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013084d3-28e1-48d8-9149-43a3cd34ed67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:52.154037 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013084d3-28e1-48d8-9149-43a3cd34ed67 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:52.175604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 902/1801] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:52 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:52.179819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f001176-79c9-4dc8-b12a-67fc67f58735 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:52.190763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f001176-79c9-4dc8-b12a-67fc67f58735 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:52.202801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af74a6-9ceb-433c-95cd-e01d93d2d4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:52.203424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af74a6-9ceb-433c-95cd-e01d93d2d4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:52.203704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af74a6-9ceb-433c-95cd-e01d93d2d4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:52.203972 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af74a6-9ceb-433c-95cd-e01d93d2d4f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:52.204621 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af74a6-9ceb-433c-95cd-e01d93d2d4f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:52.206719 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af74a6-9ceb-433c-95cd-e01d93d2d4f7 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:52.217707 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 900/1802] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:52.222085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013084d3-28e1-48d8-9149-43a3cd34ed67 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:52.232831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013084d3-28e1-48d8-9149-43a3cd34ed67 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:52.237510 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69ebc7-d055-4ec2-97d1-c0b1f9b4d41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:52.238202 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69ebc7-d055-4ec2-97d1-c0b1f9b4d41b admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:52.238484 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69ebc7-d055-4ec2-97d1-c0b1f9b4d41b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:52.238761 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69ebc7-d055-4ec2-97d1-c0b1f9b4d41b admin admin] PATH_INFO: `/v3/projects/4a57acbbf9aa4072a1511a6631b4cf79/users/96530c0a86c144e68e79a7214eb9b58b/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:52.254332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69ebc7-d055-4ec2-97d1-c0b1f9b4d41b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a57acbbf9aa4072a1511a6631b4cf79, user_id=96530c0a86c144e68e79a7214eb9b58b, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:52.256969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69ebc7-d055-4ec2-97d1-c0b1f9b4d41b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:52.269724 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 903/1803] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:52 2022] PUT /identity/v3/projects/4a57acbbf9aa4072a1511a6631b4cf79/users/96530c0a86c144e68e79a7214eb9b58b/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:52.273789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af74a6-9ceb-433c-95cd-e01d93d2d4f7 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:52.284564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af74a6-9ceb-433c-95cd-e01d93d2d4f7 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:52.346788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb73db8-b04a-4fe8-9ed3-0d8591d0c10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:52.347386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb73db8-b04a-4fe8-9ed3-0d8591d0c10f admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:52.347662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb73db8-b04a-4fe8-9ed3-0d8591d0c10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:52.347937 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb73db8-b04a-4fe8-9ed3-0d8591d0c10f admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:52.348565 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb73db8-b04a-4fe8-9ed3-0d8591d0c10f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:52.350163 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb73db8-b04a-4fe8-9ed3-0d8591d0c10f admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:52.363166 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 901/1804] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:52.364404 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69ebc7-d055-4ec2-97d1-c0b1f9b4d41b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:52.375621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69ebc7-d055-4ec2-97d1-c0b1f9b4d41b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:52.380238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc07c65-cd98-482e-b54e-77904978076c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:52.380957 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc07c65-cd98-482e-b54e-77904978076c admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:52.381234 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc07c65-cd98-482e-b54e-77904978076c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:52.381507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc07c65-cd98-482e-b54e-77904978076c admin admin] PATH_INFO: `/v3/projects/4a57acbbf9aa4072a1511a6631b4cf79/users/96530c0a86c144e68e79a7214eb9b58b/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:52.383765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc07c65-cd98-482e-b54e-77904978076c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a57acbbf9aa4072a1511a6631b4cf79, user_id=96530c0a86c144e68e79a7214eb9b58b, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:52.388323 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc07c65-cd98-482e-b54e-77904978076c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:52.402193 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 904/1805] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:52 2022] PUT /identity/v3/projects/4a57acbbf9aa4072a1511a6631b4cf79/users/96530c0a86c144e68e79a7214eb9b58b/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:52.412173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06c760-1737-472a-a2fe-52cd4d010560 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:52.415430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06c760-1737-472a-a2fe-52cd4d010560 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:52.415430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06c760-1737-472a-a2fe-52cd4d010560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:52.421974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:52.445997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-ea06c760-1737-472a-a2fe-52cd4d010560 None None] MFA Rules not processed for user `96530c0a86c144e68e79a7214eb9b58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:52.447455 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ea06c760-1737-472a-a2fe-52cd4d010560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:52.531847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 902/1806] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:52.546293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc07c65-cd98-482e-b54e-77904978076c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:52.548195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8cc07c65-cd98-482e-b54e-77904978076c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:52.607105 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc07c65-cd98-482e-b54e-77904978076c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:52.610794 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250833a-62fa-4b46-b95c-a02694ad4ba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:52.611406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250833a-62fa-4b46-b95c-a02694ad4ba4 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:52.611682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250833a-62fa-4b46-b95c-a02694ad4ba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:52.611955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250833a-62fa-4b46-b95c-a02694ad4ba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:52.622104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250833a-62fa-4b46-b95c-a02694ad4ba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:52.623555 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250833a-62fa-4b46-b95c-a02694ad4ba4 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:52.633388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 905/1807] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:53.277972 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06c760-1737-472a-a2fe-52cd4d010560 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:53.339628 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06c760-1737-472a-a2fe-52cd4d010560 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:53.344200 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babda713-dc14-4a55-a787-0546a95eeb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:53.344826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babda713-dc14-4a55-a787-0546a95eeb08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:53.345115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babda713-dc14-4a55-a787-0546a95eeb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:53.345385 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babda713-dc14-4a55-a787-0546a95eeb08 admin admin] PATH_INFO: `/v3/users/495cf187b5114f19a83fa53f11a7b1b1` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:53.346537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babda713-dc14-4a55-a787-0546a95eeb08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=495cf187b5114f19a83fa53f11a7b1b1)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:53.348459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babda713-dc14-4a55-a787-0546a95eeb08 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:53.393473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-babda713-dc14-4a55-a787-0546a95eeb08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '495cf187b5114f19a83fa53f11a7b1b1'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:53.401222 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-babda713-dc14-4a55-a787-0546a95eeb08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '495cf187b5114f19a83fa53f11a7b1b1'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:53.413410 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-babda713-dc14-4a55-a787-0546a95eeb08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '495cf187b5114f19a83fa53f11a7b1b1'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:53.422457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-babda713-dc14-4a55-a787-0546a95eeb08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '495cf187b5114f19a83fa53f11a7b1b1'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:53.423141 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-babda713-dc14-4a55-a787-0546a95eeb08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '495cf187b5114f19a83fa53f11a7b1b1'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:53.429111 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 903/1808] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:06:53 2022] DELETE /identity/v3/users/495cf187b5114f19a83fa53f11a7b1b1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:53.433631 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcdcff-737f-4169-8b25-f40243315d9c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:53.434653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcdcff-737f-4169-8b25-f40243315d9c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:53.434653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcdcff-737f-4169-8b25-f40243315d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:53.454321 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f4bcdcff-737f-4169-8b25-f40243315d9c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:53.455533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f4bcdcff-737f-4169-8b25-f40243315d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:53.486459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:53.518917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 906/1809] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:06:53.529015 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babda713-dc14-4a55-a787-0546a95eeb08 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:53.530513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-babda713-dc14-4a55-a787-0546a95eeb08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:53.542342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:53.601313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babda713-dc14-4a55-a787-0546a95eeb08 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:53.607999 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed48c4-d6db-479a-a4b1-b6028b45eb58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:53.608619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed48c4-d6db-479a-a4b1-b6028b45eb58 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:53.608902 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed48c4-d6db-479a-a4b1-b6028b45eb58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:53.609172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed48c4-d6db-479a-a4b1-b6028b45eb58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:53.619240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed48c4-d6db-479a-a4b1-b6028b45eb58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:53.620941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed48c4-d6db-479a-a4b1-b6028b45eb58 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:53.630966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 904/1810] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:54.113023 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcdcff-737f-4169-8b25-f40243315d9c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:54.114641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f4bcdcff-737f-4169-8b25-f40243315d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:54.128432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcdcff-737f-4169-8b25-f40243315d9c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:54.134841 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6378f2-cc16-4f4a-b15f-983892b8ef96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:54.135569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6378f2-cc16-4f4a-b15f-983892b8ef96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:54.135925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6378f2-cc16-4f4a-b15f-983892b8ef96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:54.136273 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6378f2-cc16-4f4a-b15f-983892b8ef96 admin admin] PATH_INFO: `/v3/projects/1bab48e5f3f34d179411e2e8ac66a982` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:54.137486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6378f2-cc16-4f4a-b15f-983892b8ef96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bab48e5f3f34d179411e2e8ac66a982)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:54.139680 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6378f2-cc16-4f4a-b15f-983892b8ef96 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:54.195775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc6378f2-cc16-4f4a-b15f-983892b8ef96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bab48e5f3f34d179411e2e8ac66a982', 'request_id': 'req-cc6378f2-cc16-4f4a-b15f-983892b8ef96'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:54.202916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc6378f2-cc16-4f4a-b15f-983892b8ef96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bab48e5f3f34d179411e2e8ac66a982', 'request_id': 'req-cc6378f2-cc16-4f4a-b15f-983892b8ef96'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:54.208693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc6378f2-cc16-4f4a-b15f-983892b8ef96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bab48e5f3f34d179411e2e8ac66a982', 'request_id': 'req-cc6378f2-cc16-4f4a-b15f-983892b8ef96'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:54.220181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc6378f2-cc16-4f4a-b15f-983892b8ef96 admin admin] The token cache is being invalidate because project 1bab48e5f3f34d179411e2e8ac66a982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:06:54.220470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc6378f2-cc16-4f4a-b15f-983892b8ef96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:54.221122 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc6378f2-cc16-4f4a-b15f-983892b8ef96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:54.222673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 907/1811] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:06:54 2022] DELETE /identity/v3/projects/1bab48e5f3f34d179411e2e8ac66a982 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:54.226834 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed48c4-d6db-479a-a4b1-b6028b45eb58 service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:54.237626 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed48c4-d6db-479a-a4b1-b6028b45eb58 service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:54.295268 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702c686-4b49-4418-bdd4-ebe672aaa9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:54.295856 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702c686-4b49-4418-bdd4-ebe672aaa9c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:54.296154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702c686-4b49-4418-bdd4-ebe672aaa9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:54.296447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702c686-4b49-4418-bdd4-ebe672aaa9c2 admin admin] PATH_INFO: `/v3/users/96530c0a86c144e68e79a7214eb9b58b` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:54.297771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702c686-4b49-4418-bdd4-ebe672aaa9c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96530c0a86c144e68e79a7214eb9b58b)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:54.299889 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702c686-4b49-4418-bdd4-ebe672aaa9c2 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:54.354458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4702c686-4b49-4418-bdd4-ebe672aaa9c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96530c0a86c144e68e79a7214eb9b58b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:54.366193 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4702c686-4b49-4418-bdd4-ebe672aaa9c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96530c0a86c144e68e79a7214eb9b58b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:54.381729 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4702c686-4b49-4418-bdd4-ebe672aaa9c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96530c0a86c144e68e79a7214eb9b58b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:54.391237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4702c686-4b49-4418-bdd4-ebe672aaa9c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96530c0a86c144e68e79a7214eb9b58b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:54.392131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4702c686-4b49-4418-bdd4-ebe672aaa9c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96530c0a86c144e68e79a7214eb9b58b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:54.401887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 905/1812] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:06:54 2022] DELETE /identity/v3/users/96530c0a86c144e68e79a7214eb9b58b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:06:54.690085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6378f2-cc16-4f4a-b15f-983892b8ef96 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:54.692361 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cc6378f2-cc16-4f4a-b15f-983892b8ef96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:54.717604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6378f2-cc16-4f4a-b15f-983892b8ef96 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:54.776764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e8f37-8457-465b-a7f7-ff6148b60735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:54.777571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e8f37-8457-465b-a7f7-ff6148b60735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:54.777903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e8f37-8457-465b-a7f7-ff6148b60735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:54.778208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e8f37-8457-465b-a7f7-ff6148b60735 admin admin] PATH_INFO: `/v3/projects/4a57acbbf9aa4072a1511a6631b4cf79` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:54.780116 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e8f37-8457-465b-a7f7-ff6148b60735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a57acbbf9aa4072a1511a6631b4cf79)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:54.782020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e8f37-8457-465b-a7f7-ff6148b60735 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:54.829882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8a0e8f37-8457-465b-a7f7-ff6148b60735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a57acbbf9aa4072a1511a6631b4cf79', 'request_id': 'req-8a0e8f37-8457-465b-a7f7-ff6148b60735'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:54.838184 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8a0e8f37-8457-465b-a7f7-ff6148b60735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a57acbbf9aa4072a1511a6631b4cf79', 'request_id': 'req-8a0e8f37-8457-465b-a7f7-ff6148b60735'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:54.842461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:54.844512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8a0e8f37-8457-465b-a7f7-ff6148b60735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a57acbbf9aa4072a1511a6631b4cf79', 'request_id': 'req-8a0e8f37-8457-465b-a7f7-ff6148b60735'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:54.862968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8a0e8f37-8457-465b-a7f7-ff6148b60735 admin admin] The token cache is being invalidate because project 4a57acbbf9aa4072a1511a6631b4cf79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:06:54.863340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8a0e8f37-8457-465b-a7f7-ff6148b60735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:54.864130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8a0e8f37-8457-465b-a7f7-ff6148b60735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:54.865612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 908/1813] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:06:54 2022] DELETE /identity/v3/projects/4a57acbbf9aa4072a1511a6631b4cf79 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:54.871510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cad76-7591-4825-83bd-47e70ec04c99 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:54.871798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cad76-7591-4825-83bd-47e70ec04c99 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:54.872081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cad76-7591-4825-83bd-47e70ec04c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:54.894343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-fb0cad76-7591-4825-83bd-47e70ec04c99 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:54.895571 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-fb0cad76-7591-4825-83bd-47e70ec04c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:54.923196 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:54.969538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 906/1814] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:54.980204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95703e76-d87f-47b2-bb1b-7a95e8d66a50 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:54.980483 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95703e76-d87f-47b2-bb1b-7a95e8d66a50 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:54.980768 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95703e76-d87f-47b2-bb1b-7a95e8d66a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:55.001187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-95703e76-d87f-47b2-bb1b-7a95e8d66a50 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:55.002414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-95703e76-d87f-47b2-bb1b-7a95e8d66a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:55.011633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 909/1815] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:55.016001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cad76-7591-4825-83bd-47e70ec04c99 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:55.025676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cad76-7591-4825-83bd-47e70ec04c99 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:55.030252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3b228-77cd-45c2-8de8-8b1267df15e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:55.030851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3b228-77cd-45c2-8de8-8b1267df15e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:55.031132 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3b228-77cd-45c2-8de8-8b1267df15e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:55.031396 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3b228-77cd-45c2-8de8-8b1267df15e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:55.032040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3b228-77cd-45c2-8de8-8b1267df15e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:55.032452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3b228-77cd-45c2-8de8-8b1267df15e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:55.034985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3b228-77cd-45c2-8de8-8b1267df15e8 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:55.045992 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 907/1816] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:06:55.050384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95703e76-d87f-47b2-bb1b-7a95e8d66a50 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:55.059992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95703e76-d87f-47b2-bb1b-7a95e8d66a50 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:55.064914 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e277247-ed35-4526-bd31-278dd4acc77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:55.065504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e277247-ed35-4526-bd31-278dd4acc77d admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:55.065778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e277247-ed35-4526-bd31-278dd4acc77d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:55.066046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e277247-ed35-4526-bd31-278dd4acc77d admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:55.066688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e277247-ed35-4526-bd31-278dd4acc77d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:55.068579 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e277247-ed35-4526-bd31-278dd4acc77d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:55.096411 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 910/1817] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:55 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:55.100482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3b228-77cd-45c2-8de8-8b1267df15e8 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:55.110521 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3b228-77cd-45c2-8de8-8b1267df15e8 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:55.169304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42373b0a-2c11-4208-a83e-11f0c85c0755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:55.169728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42373b0a-2c11-4208-a83e-11f0c85c0755 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:55.170154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42373b0a-2c11-4208-a83e-11f0c85c0755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:55.170422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42373b0a-2c11-4208-a83e-11f0c85c0755 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:55.171077 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42373b0a-2c11-4208-a83e-11f0c85c0755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:55.173478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42373b0a-2c11-4208-a83e-11f0c85c0755 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:55.202402 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 908/1818] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:55 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:55.207354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e277247-ed35-4526-bd31-278dd4acc77d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:55.221169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e277247-ed35-4526-bd31-278dd4acc77d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:55.226979 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39128f-072d-428d-a852-f2bb2b4b20e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:55.227148 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39128f-072d-428d-a852-f2bb2b4b20e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:55.227928 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39128f-072d-428d-a852-f2bb2b4b20e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:55.228195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39128f-072d-428d-a852-f2bb2b4b20e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:55.228835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39128f-072d-428d-a852-f2bb2b4b20e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:55.230438 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39128f-072d-428d-a852-f2bb2b4b20e5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:55.240709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 911/1819] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:55.244758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42373b0a-2c11-4208-a83e-11f0c85c0755 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:55.255084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42373b0a-2c11-4208-a83e-11f0c85c0755 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:55.260191 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6656c3-5fe8-45a6-9a40-885c6f38d680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:55.260782 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6656c3-5fe8-45a6-9a40-885c6f38d680 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:55.261063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6656c3-5fe8-45a6-9a40-885c6f38d680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:55.261337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6656c3-5fe8-45a6-9a40-885c6f38d680 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:55.261978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6656c3-5fe8-45a6-9a40-885c6f38d680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:55.263608 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6656c3-5fe8-45a6-9a40-885c6f38d680 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:55.275075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 909/1820] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:55.279080 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39128f-072d-428d-a852-f2bb2b4b20e5 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:55.289405 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39128f-072d-428d-a852-f2bb2b4b20e5 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:55.294311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84116512-4007-4707-8e5d-70e2cf1611c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:55.294971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84116512-4007-4707-8e5d-70e2cf1611c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:55.295250 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84116512-4007-4707-8e5d-70e2cf1611c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:55.295523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84116512-4007-4707-8e5d-70e2cf1611c7 admin admin] PATH_INFO: `/v3/projects/3458ea8e53454989a11a0eebd9d26e66/users/095270e399c64de1911733ff2940cfff/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:55.310178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84116512-4007-4707-8e5d-70e2cf1611c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3458ea8e53454989a11a0eebd9d26e66, user_id=095270e399c64de1911733ff2940cfff, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:55.312813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84116512-4007-4707-8e5d-70e2cf1611c7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:55.326100 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 912/1821] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:55 2022] PUT /identity/v3/projects/3458ea8e53454989a11a0eebd9d26e66/users/095270e399c64de1911733ff2940cfff/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:55.331328 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48c4ce-7da7-446b-a8e1-75ba0bed303e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:55.331778 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48c4ce-7da7-446b-a8e1-75ba0bed303e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:55.331889 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48c4ce-7da7-446b-a8e1-75ba0bed303e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:55.351058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-6a48c4ce-7da7-446b-a8e1-75ba0bed303e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:55.352194 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6a48c4ce-7da7-446b-a8e1-75ba0bed303e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:55.410472 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 910/1822] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:55.420185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84116512-4007-4707-8e5d-70e2cf1611c7 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:55.421891 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-84116512-4007-4707-8e5d-70e2cf1611c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:55.479704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84116512-4007-4707-8e5d-70e2cf1611c7 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:55.483286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23323221-caf1-4a65-b57d-1556b9efa081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:55.483890 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23323221-caf1-4a65-b57d-1556b9efa081 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:55.484165 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23323221-caf1-4a65-b57d-1556b9efa081 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:55.484434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23323221-caf1-4a65-b57d-1556b9efa081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:55.494375 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23323221-caf1-4a65-b57d-1556b9efa081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:55.496200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23323221-caf1-4a65-b57d-1556b9efa081 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:55.506089 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 913/1823] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:56.279349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1b85d-9893-4480-a2ea-e4dd18fde74a None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:56.279640 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1b85d-9893-4480-a2ea-e4dd18fde74a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:56.280031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1b85d-9893-4480-a2ea-e4dd18fde74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:56.326699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a9f1b85d-9893-4480-a2ea-e4dd18fde74a None None] MFA Rules not processed for user `095270e399c64de1911733ff2940cfff`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:56.328048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a9f1b85d-9893-4480-a2ea-e4dd18fde74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:56.365189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:56.430159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 911/1824] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:56.432793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23323221-caf1-4a65-b57d-1556b9efa081 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:56.443462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23323221-caf1-4a65-b57d-1556b9efa081 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:56.448361 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b04de7-dde8-4f6b-950d-d6d25af68fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:56.449065 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b04de7-dde8-4f6b-950d-d6d25af68fae admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:56.449408 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b04de7-dde8-4f6b-950d-d6d25af68fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:56.449844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b04de7-dde8-4f6b-950d-d6d25af68fae admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:56.451110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b04de7-dde8-4f6b-950d-d6d25af68fae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:56.453040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b04de7-dde8-4f6b-950d-d6d25af68fae admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:56.483691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 914/1825] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:56 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:56.487966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1b85d-9893-4480-a2ea-e4dd18fde74a None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:56.502491 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1b85d-9893-4480-a2ea-e4dd18fde74a None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:56.571927 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620007bf-ce6b-4427-a68b-849e272559cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:56.573179 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620007bf-ce6b-4427-a68b-849e272559cd admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:56.573179 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620007bf-ce6b-4427-a68b-849e272559cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:56.573508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620007bf-ce6b-4427-a68b-849e272559cd admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:56.574548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620007bf-ce6b-4427-a68b-849e272559cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:56.576871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620007bf-ce6b-4427-a68b-849e272559cd admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:56.607362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 912/1826] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:56 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:56.611535 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b04de7-dde8-4f6b-950d-d6d25af68fae admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:56.622722 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b04de7-dde8-4f6b-950d-d6d25af68fae admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:56.627573 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f02c21-0a16-4466-8f7f-ee88cbf0212e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:56.628261 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f02c21-0a16-4466-8f7f-ee88cbf0212e admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:56.628604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f02c21-0a16-4466-8f7f-ee88cbf0212e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:56.628991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f02c21-0a16-4466-8f7f-ee88cbf0212e admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:56.629689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f02c21-0a16-4466-8f7f-ee88cbf0212e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:56.631416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f02c21-0a16-4466-8f7f-ee88cbf0212e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:56.651865 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 915/1827] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:56.654786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620007bf-ce6b-4427-a68b-849e272559cd admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:56.666095 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620007bf-ce6b-4427-a68b-849e272559cd admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:56.671110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c2330-ac8b-4270-a679-f4eb2720e6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:56.671789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c2330-ac8b-4270-a679-f4eb2720e6e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:56.672122 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c2330-ac8b-4270-a679-f4eb2720e6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:56.672399 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c2330-ac8b-4270-a679-f4eb2720e6e9 admin admin] PATH_INFO: `/v3/projects/5eccee94872840afacff642b7d05cb72/users/6e638bbbe18f432392f34a4cd33edbd7/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:56.690719 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c2330-ac8b-4270-a679-f4eb2720e6e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eccee94872840afacff642b7d05cb72, user_id=6e638bbbe18f432392f34a4cd33edbd7, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:56.693523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c2330-ac8b-4270-a679-f4eb2720e6e9 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:56.710650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 913/1828] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:56 2022] PUT /identity/v3/projects/5eccee94872840afacff642b7d05cb72/users/6e638bbbe18f432392f34a4cd33edbd7/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:56.715231 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f02c21-0a16-4466-8f7f-ee88cbf0212e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:56.732117 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f02c21-0a16-4466-8f7f-ee88cbf0212e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:56.795830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:56.807830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a2099-ab99-4518-9801-526bf247cf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:56.808574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a2099-ab99-4518-9801-526bf247cf1d admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:56.808926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a2099-ab99-4518-9801-526bf247cf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:56.809265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a2099-ab99-4518-9801-526bf247cf1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:56.810023 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a2099-ab99-4518-9801-526bf247cf1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:56.811791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a2099-ab99-4518-9801-526bf247cf1d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:56.822549 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 916/1829] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:06:56.826731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c2330-ac8b-4270-a679-f4eb2720e6e9 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:56.837604 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c2330-ac8b-4270-a679-f4eb2720e6e9 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:56.842213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654489a3-913c-4889-93b6-e88b65b23be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:56.842856 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654489a3-913c-4889-93b6-e88b65b23be6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:56.843134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654489a3-913c-4889-93b6-e88b65b23be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:56.843403 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654489a3-913c-4889-93b6-e88b65b23be6 admin admin] PATH_INFO: `/v3/projects/5eccee94872840afacff642b7d05cb72/users/6e638bbbe18f432392f34a4cd33edbd7/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:56.845304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654489a3-913c-4889-93b6-e88b65b23be6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eccee94872840afacff642b7d05cb72, user_id=6e638bbbe18f432392f34a4cd33edbd7, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:56.847978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654489a3-913c-4889-93b6-e88b65b23be6 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:56.858637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 914/1830] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:56 2022] PUT /identity/v3/projects/5eccee94872840afacff642b7d05cb72/users/6e638bbbe18f432392f34a4cd33edbd7/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:57.484300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a2099-ab99-4518-9801-526bf247cf1d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:57.486020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1c7a2099-ab99-4518-9801-526bf247cf1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:57.507748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a2099-ab99-4518-9801-526bf247cf1d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:57.635327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd1ca1-9841-4a35-b810-598531f006b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:57.636742 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd1ca1-9841-4a35-b810-598531f006b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:57.637795 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd1ca1-9841-4a35-b810-598531f006b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:57.638768 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd1ca1-9841-4a35-b810-598531f006b9 admin admin] PATH_INFO: `/v3/users/6ef4b0ce04da4e969efab2861fb6bf46` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:57.640859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd1ca1-9841-4a35-b810-598531f006b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ef4b0ce04da4e969efab2861fb6bf46)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:57.643203 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd1ca1-9841-4a35-b810-598531f006b9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:57.715357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c3cd1ca1-9841-4a35-b810-598531f006b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ef4b0ce04da4e969efab2861fb6bf46'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:57.716570 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c3cd1ca1-9841-4a35-b810-598531f006b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ef4b0ce04da4e969efab2861fb6bf46'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:57.734178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c3cd1ca1-9841-4a35-b810-598531f006b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ef4b0ce04da4e969efab2861fb6bf46'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:57.745761 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c3cd1ca1-9841-4a35-b810-598531f006b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ef4b0ce04da4e969efab2861fb6bf46'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:57.765935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c3cd1ca1-9841-4a35-b810-598531f006b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ef4b0ce04da4e969efab2861fb6bf46'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:57.776909 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 917/1831] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:06:57 2022] DELETE /identity/v3/users/6ef4b0ce04da4e969efab2861fb6bf46 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:57.778731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a4b31-d2e5-4f38-84dc-f3b07cfe50fc None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:57.779748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a4b31-d2e5-4f38-84dc-f3b07cfe50fc None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:57.780219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a4b31-d2e5-4f38-84dc-f3b07cfe50fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:57.833663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-2d3a4b31-d2e5-4f38-84dc-f3b07cfe50fc None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:57.835703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2d3a4b31-d2e5-4f38-84dc-f3b07cfe50fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:57.923800 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 915/1832] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:06:57.983197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd1ca1-9841-4a35-b810-598531f006b9 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:57.985347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c3cd1ca1-9841-4a35-b810-598531f006b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:58.113751 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd1ca1-9841-4a35-b810-598531f006b9 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:58.141292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d564c-74a5-4baa-b078-b773f4e7fc17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:58.141292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d564c-74a5-4baa-b078-b773f4e7fc17 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:58.141292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d564c-74a5-4baa-b078-b773f4e7fc17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:58.141292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d564c-74a5-4baa-b078-b773f4e7fc17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:58.166259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d564c-74a5-4baa-b078-b773f4e7fc17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:58.168397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d564c-74a5-4baa-b078-b773f4e7fc17 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:58.183705 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 918/1833] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:58.409286 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ba0fe-e5ba-47b1-acdd-86ff517b5288 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:58.409578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ba0fe-e5ba-47b1-acdd-86ff517b5288 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:58.409850 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ba0fe-e5ba-47b1-acdd-86ff517b5288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:58.445369 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a23ba0fe-e5ba-47b1-acdd-86ff517b5288 None None] MFA Rules not processed for user `6e638bbbe18f432392f34a4cd33edbd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:58.447284 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a23ba0fe-e5ba-47b1-acdd-86ff517b5288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:58.453996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:58.550288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 916/1834] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:58.560747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d564c-74a5-4baa-b078-b773f4e7fc17 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:58.562293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c30d564c-74a5-4baa-b078-b773f4e7fc17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:58.582476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:58.656115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ba0fe-e5ba-47b1-acdd-86ff517b5288 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:58.657730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a23ba0fe-e5ba-47b1-acdd-86ff517b5288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:58.676963 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ba0fe-e5ba-47b1-acdd-86ff517b5288 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:58.684011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0000d-33d1-464a-bd38-553d7204f326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:58.684802 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0000d-33d1-464a-bd38-553d7204f326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:58.685083 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0000d-33d1-464a-bd38-553d7204f326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:58.685368 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0000d-33d1-464a-bd38-553d7204f326 admin admin] PATH_INFO: `/v3/projects/a88dca3f1f934b2e812a1ffd3977b480` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:58.692317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d564c-74a5-4baa-b078-b773f4e7fc17 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:58.696382 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03ddd6-cb7a-485a-a0dc-0018a40d08f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:58.696996 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03ddd6-cb7a-485a-a0dc-0018a40d08f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:58.697278 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03ddd6-cb7a-485a-a0dc-0018a40d08f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:58.697627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03ddd6-cb7a-485a-a0dc-0018a40d08f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:58.699248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0000d-33d1-464a-bd38-553d7204f326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a88dca3f1f934b2e812a1ffd3977b480)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:58.699448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ba03ddd6-cb7a-485a-a0dc-0018a40d08f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:58.701620 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0000d-33d1-464a-bd38-553d7204f326 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:58.757725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2db0000d-33d1-464a-bd38-553d7204f326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a88dca3f1f934b2e812a1ffd3977b480', 'request_id': 'req-2db0000d-33d1-464a-bd38-553d7204f326'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:58.765316 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2db0000d-33d1-464a-bd38-553d7204f326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a88dca3f1f934b2e812a1ffd3977b480', 'request_id': 'req-2db0000d-33d1-464a-bd38-553d7204f326'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:58.768176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03ddd6-cb7a-485a-a0dc-0018a40d08f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:58.769880 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03ddd6-cb7a-485a-a0dc-0018a40d08f2 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:58.771692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2db0000d-33d1-464a-bd38-553d7204f326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a88dca3f1f934b2e812a1ffd3977b480', 'request_id': 'req-2db0000d-33d1-464a-bd38-553d7204f326'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:58.782907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 919/1835] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:58.786243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2db0000d-33d1-464a-bd38-553d7204f326 admin admin] The token cache is being invalidate because project a88dca3f1f934b2e812a1ffd3977b480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:06:58.786824 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2db0000d-33d1-464a-bd38-553d7204f326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:58.787398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2db0000d-33d1-464a-bd38-553d7204f326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:06:58.793569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 917/1836] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:06:58 2022] DELETE /identity/v3/projects/a88dca3f1f934b2e812a1ffd3977b480 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:58.795534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce474459-e7c4-45f3-b0a0-e806c3e12e7a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:58.796017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce474459-e7c4-45f3-b0a0-e806c3e12e7a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:58.796524 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce474459-e7c4-45f3-b0a0-e806c3e12e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:58.818261 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ce474459-e7c4-45f3-b0a0-e806c3e12e7a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:58.819573 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ce474459-e7c4-45f3-b0a0-e806c3e12e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:58.905911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc883fe8-f0f8-4ac4-b204-20d3a6ce1a43 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:58.906226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc883fe8-f0f8-4ac4-b204-20d3a6ce1a43 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:58.906500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc883fe8-f0f8-4ac4-b204-20d3a6ce1a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:58.910308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 920/1837] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:58.928496 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-bc883fe8-f0f8-4ac4-b204-20d3a6ce1a43 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:58.930930 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-bc883fe8-f0f8-4ac4-b204-20d3a6ce1a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:58.940995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 918/1838] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:58.945136 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce474459-e7c4-45f3-b0a0-e806c3e12e7a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:58.955544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce474459-e7c4-45f3-b0a0-e806c3e12e7a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:58.960784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a8fd4-46c4-4ecd-b82b-30d8aaa2eb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:58.961565 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a8fd4-46c4-4ecd-b82b-30d8aaa2eb1c admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:58.961949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a8fd4-46c4-4ecd-b82b-30d8aaa2eb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:58.962537 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a8fd4-46c4-4ecd-b82b-30d8aaa2eb1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:58.965909 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a8fd4-46c4-4ecd-b82b-30d8aaa2eb1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:06:58.965909 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a8fd4-46c4-4ecd-b82b-30d8aaa2eb1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:58.965909 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a8fd4-46c4-4ecd-b82b-30d8aaa2eb1c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:58.986588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 921/1839] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:06:58.991397 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc883fe8-f0f8-4ac4-b204-20d3a6ce1a43 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:59.003537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc883fe8-f0f8-4ac4-b204-20d3a6ce1a43 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:59.010328 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe540f-ca21-4287-aef4-a2dddd71be11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:59.010982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe540f-ca21-4287-aef4-a2dddd71be11 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:59.011235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe540f-ca21-4287-aef4-a2dddd71be11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:59.011500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe540f-ca21-4287-aef4-a2dddd71be11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:59.012136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe540f-ca21-4287-aef4-a2dddd71be11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:59.014388 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe540f-ca21-4287-aef4-a2dddd71be11 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:59.046151 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 919/1840] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:58 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:06:59.048804 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a8fd4-46c4-4ecd-b82b-30d8aaa2eb1c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:59.058875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a8fd4-46c4-4ecd-b82b-30d8aaa2eb1c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:59.130421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:59.143974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae76e0-9579-4a34-b60f-343da575b7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:59.144612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae76e0-9579-4a34-b60f-343da575b7ca admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:59.144889 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae76e0-9579-4a34-b60f-343da575b7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:59.145162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae76e0-9579-4a34-b60f-343da575b7ca admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:59.145936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae76e0-9579-4a34-b60f-343da575b7ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:59.148027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae76e0-9579-4a34-b60f-343da575b7ca admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:59.205650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 922/1841] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:59 2022] POST /identity/v3/users => generated 282 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:59.211074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe540f-ca21-4287-aef4-a2dddd71be11 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:59.222750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe540f-ca21-4287-aef4-a2dddd71be11 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:59.231641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b22c3-b494-4342-aa25-08e34ac4ab8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:59.232337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b22c3-b494-4342-aa25-08e34ac4ab8c admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:59.232730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b22c3-b494-4342-aa25-08e34ac4ab8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:59.233216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b22c3-b494-4342-aa25-08e34ac4ab8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:59.233947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b22c3-b494-4342-aa25-08e34ac4ab8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:59.235696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b22c3-b494-4342-aa25-08e34ac4ab8c admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:59.248130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 920/1842] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:59.252295 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae76e0-9579-4a34-b60f-343da575b7ca admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:59.264734 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae76e0-9579-4a34-b60f-343da575b7ca admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:59.269442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364bd1a-1ca8-40d7-a1bb-b797d42c7d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:59.270174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364bd1a-1ca8-40d7-a1bb-b797d42c7d1d admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:59.270459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364bd1a-1ca8-40d7-a1bb-b797d42c7d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:59.270730 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364bd1a-1ca8-40d7-a1bb-b797d42c7d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:59.271383 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364bd1a-1ca8-40d7-a1bb-b797d42c7d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:59.272962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364bd1a-1ca8-40d7-a1bb-b797d42c7d1d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:59.285074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 923/1843] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:59.288831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b22c3-b494-4342-aa25-08e34ac4ab8c admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:59.299889 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b22c3-b494-4342-aa25-08e34ac4ab8c admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:59.306233 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8d737-86fa-4ae1-a62e-966c6f1f7512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:59.307074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8d737-86fa-4ae1-a62e-966c6f1f7512 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:59.307397 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8d737-86fa-4ae1-a62e-966c6f1f7512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:59.307812 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8d737-86fa-4ae1-a62e-966c6f1f7512 admin admin] PATH_INFO: `/v3/projects/a1092f5d05ee45d88eac9697f1950e5f/users/b0effeb7060e43d88390e4f49fbc8c0f/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:59.324832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8d737-86fa-4ae1-a62e-966c6f1f7512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1092f5d05ee45d88eac9697f1950e5f, user_id=b0effeb7060e43d88390e4f49fbc8c0f, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:59.328269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8d737-86fa-4ae1-a62e-966c6f1f7512 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:59.349160 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 921/1844] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:59 2022] PUT /identity/v3/projects/a1092f5d05ee45d88eac9697f1950e5f/users/b0effeb7060e43d88390e4f49fbc8c0f/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:59.361736 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fd1ec-96c4-4e04-8cad-ec1fc54e6e1e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:59.362024 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fd1ec-96c4-4e04-8cad-ec1fc54e6e1e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:59.362426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fd1ec-96c4-4e04-8cad-ec1fc54e6e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:59.392697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-232fd1ec-96c4-4e04-8cad-ec1fc54e6e1e None None] MFA Rules not processed for user `b0effeb7060e43d88390e4f49fbc8c0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:59.393683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-232fd1ec-96c4-4e04-8cad-ec1fc54e6e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:59.480358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 924/1845] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:06:59.488783 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8d737-86fa-4ae1-a62e-966c6f1f7512 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:59.520335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:59.566660 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8d737-86fa-4ae1-a62e-966c6f1f7512 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:59.571549 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d71da-e2b1-41f9-b881-e79d8f26f2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:59.572170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d71da-e2b1-41f9-b881-e79d8f26f2a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:59.572452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d71da-e2b1-41f9-b881-e79d8f26f2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:59.572738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d71da-e2b1-41f9-b881-e79d8f26f2a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:59.573382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d71da-e2b1-41f9-b881-e79d8f26f2a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:59.575336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d71da-e2b1-41f9-b881-e79d8f26f2a0 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:59.610152 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 922/1846] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:06:59 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:06:59.614232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fd1ec-96c4-4e04-8cad-ec1fc54e6e1e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:59.625523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fd1ec-96c4-4e04-8cad-ec1fc54e6e1e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:59.685725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57deb7-6fb3-49b5-88d5-992230131b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:59.686434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57deb7-6fb3-49b5-88d5-992230131b6e admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:59.686711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57deb7-6fb3-49b5-88d5-992230131b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:59.686985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57deb7-6fb3-49b5-88d5-992230131b6e admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:59.687804 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57deb7-6fb3-49b5-88d5-992230131b6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:59.689824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57deb7-6fb3-49b5-88d5-992230131b6e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:59.722235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 925/1847] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:06:59 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:06:59.723728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d71da-e2b1-41f9-b881-e79d8f26f2a0 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:59.746528 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d71da-e2b1-41f9-b881-e79d8f26f2a0 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:59.752822 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8ded3-acf7-45b3-9b10-7f24c296948a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:59.753632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8ded3-acf7-45b3-9b10-7f24c296948a admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:59.754031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8ded3-acf7-45b3-9b10-7f24c296948a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:59.754405 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8ded3-acf7-45b3-9b10-7f24c296948a admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:59.755337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea8ded3-acf7-45b3-9b10-7f24c296948a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:59.757221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea8ded3-acf7-45b3-9b10-7f24c296948a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:59.770162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 923/1848] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:59.775016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57deb7-6fb3-49b5-88d5-992230131b6e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:59.788638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57deb7-6fb3-49b5-88d5-992230131b6e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:59.793646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48738364-607c-4be5-ae25-bdeac67e79ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:59.794521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48738364-607c-4be5-ae25-bdeac67e79ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:59.794864 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48738364-607c-4be5-ae25-bdeac67e79ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:59.795143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48738364-607c-4be5-ae25-bdeac67e79ef admin admin] PATH_INFO: `/v3/projects/3dfa9442af404ff681936b42e0c86f15/users/c90b8a32a236468d9ca58260d0d6b94d/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:59.810209 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48738364-607c-4be5-ae25-bdeac67e79ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfa9442af404ff681936b42e0c86f15, user_id=c90b8a32a236468d9ca58260d0d6b94d, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:59.812922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48738364-607c-4be5-ae25-bdeac67e79ef admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:59.829526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 926/1849] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:59 2022] PUT /identity/v3/projects/3dfa9442af404ff681936b42e0c86f15/users/c90b8a32a236468d9ca58260d0d6b94d/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:59.833584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8ded3-acf7-45b3-9b10-7f24c296948a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:59.844725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8ded3-acf7-45b3-9b10-7f24c296948a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:59.910848 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af5441-7464-400a-ba4f-d1b3d367bd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:59.911516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af5441-7464-400a-ba4f-d1b3d367bd3d admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:59.911837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af5441-7464-400a-ba4f-d1b3d367bd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:59.912138 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af5441-7464-400a-ba4f-d1b3d367bd3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:59.912887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af5441-7464-400a-ba4f-d1b3d367bd3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:59.914557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af5441-7464-400a-ba4f-d1b3d367bd3d admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:59.929249 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 924/1850] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:06:59.930796 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48738364-607c-4be5-ae25-bdeac67e79ef admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:59.943320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48738364-607c-4be5-ae25-bdeac67e79ef admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:59.949442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1654b-454e-45ab-a909-d3bc36e052c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:59.950143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1654b-454e-45ab-a909-d3bc36e052c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:59.950467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1654b-454e-45ab-a909-d3bc36e052c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:59.950745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1654b-454e-45ab-a909-d3bc36e052c9 admin admin] PATH_INFO: `/v3/projects/3dfa9442af404ff681936b42e0c86f15/users/c90b8a32a236468d9ca58260d0d6b94d/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:59.952669 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1654b-454e-45ab-a909-d3bc36e052c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfa9442af404ff681936b42e0c86f15, user_id=c90b8a32a236468d9ca58260d0d6b94d, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:59.955436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1654b-454e-45ab-a909-d3bc36e052c9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:59.967231 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 927/1851] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:06:59 2022] PUT /identity/v3/projects/3dfa9442af404ff681936b42e0c86f15/users/c90b8a32a236468d9ca58260d0d6b94d/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:06:59.978011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dac6d7-a68c-4995-af3e-d90d86a05943 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:59.978332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dac6d7-a68c-4995-af3e-d90d86a05943 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:59.978616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dac6d7-a68c-4995-af3e-d90d86a05943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:00.008985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-10dac6d7-a68c-4995-af3e-d90d86a05943 None None] MFA Rules not processed for user `c90b8a32a236468d9ca58260d0d6b94d`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:00.010468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-10dac6d7-a68c-4995-af3e-d90d86a05943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:00.090246 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 925/1852] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:06:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:00.093605 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9dff2-fd3e-4abe-abfa-c60d47153e58 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:00.094031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9dff2-fd3e-4abe-abfa-c60d47153e58 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:00.094419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9dff2-fd3e-4abe-abfa-c60d47153e58 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:00.096470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 928/1853] 10.222.0.62 () {54 vars in 921 bytes} [Fri Sep 9 12:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:00.101518 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dac6d7-a68c-4995-af3e-d90d86a05943 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:00.102968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae9d68-899f-4f89-ac67-941ea267abfe None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:00.103797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae9d68-899f-4f89-ac67-941ea267abfe None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:00.103797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae9d68-899f-4f89-ac67-941ea267abfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:00.103964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-10dac6d7-a68c-4995-af3e-d90d86a05943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:00.124645 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f8ae9d68-899f-4f89-ac67-941ea267abfe None None] MFA Rules not processed for user `1b30e80c421245b5a1ed5849b28ea90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:00.126163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f8ae9d68-899f-4f89-ac67-941ea267abfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:00.127958 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:00.137168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:00.176577 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dac6d7-a68c-4995-af3e-d90d86a05943 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:00.180122 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d986744-3b44-438a-9751-ad0367899d28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:00.180788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d986744-3b44-438a-9751-ad0367899d28 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:00.181064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d986744-3b44-438a-9751-ad0367899d28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:00.181339 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d986744-3b44-438a-9751-ad0367899d28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:00.190417 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d986744-3b44-438a-9751-ad0367899d28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:00.197248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d986744-3b44-438a-9751-ad0367899d28 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:00.207792 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 926/1854] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:00.208453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 929/1855] 10.222.0.62 () {58 vars in 1018 bytes} [Fri Sep 9 12:07:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:07:00.229157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d986744-3b44-438a-9751-ad0367899d28 service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:00.230766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-5d986744-3b44-438a-9751-ad0367899d28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:00.250558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d986744-3b44-438a-9751-ad0367899d28 service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:00.272140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd9deb-b91a-4e81-9016-474e6ff34f68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:00.273574 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd9deb-b91a-4e81-9016-474e6ff34f68 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:00.275270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd9deb-b91a-4e81-9016-474e6ff34f68 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:00.275706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd9deb-b91a-4e81-9016-474e6ff34f68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:00.292072 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd9deb-b91a-4e81-9016-474e6ff34f68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:00.295726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd9deb-b91a-4e81-9016-474e6ff34f68 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:00.303118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae9d68-899f-4f89-ac67-941ea267abfe None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:00.304654 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f8ae9d68-899f-4f89-ac67-941ea267abfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:00.317506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 927/1856] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:00.331800 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae9d68-899f-4f89-ac67-941ea267abfe None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:00.348238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95946bb2-680d-4c45-b39e-ed19e27c6fd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:00.349150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95946bb2-680d-4c45-b39e-ed19e27c6fd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:00.349744 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95946bb2-680d-4c45-b39e-ed19e27c6fd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:00.349744 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95946bb2-680d-4c45-b39e-ed19e27c6fd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:00.363898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-95946bb2-680d-4c45-b39e-ed19e27c6fd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:00.389583 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd9deb-b91a-4e81-9016-474e6ff34f68 service nova] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:00.404581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd9deb-b91a-4e81-9016-474e6ff34f68 service nova] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:00.412023 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e24b78-a795-4aa6-87eb-8462e607de60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:00.412652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e24b78-a795-4aa6-87eb-8462e607de60 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:00.412917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e24b78-a795-4aa6-87eb-8462e607de60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:00.413195 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e24b78-a795-4aa6-87eb-8462e607de60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:00.514165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e24b78-a795-4aa6-87eb-8462e607de60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:00.515839 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e24b78-a795-4aa6-87eb-8462e607de60 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:00.530084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 928/1857] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:07:00.560033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95946bb2-680d-4c45-b39e-ed19e27c6fd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:00.561499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95946bb2-680d-4c45-b39e-ed19e27c6fd6 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:00.601892 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 930/1858] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:07:10.523499 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=52 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:10.583980 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=54 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:14.694008 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e24b78-a795-4aa6-87eb-8462e607de60 service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:14.697723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e3e24b78-a795-4aa6-87eb-8462e607de60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:14.827606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e24b78-a795-4aa6-87eb-8462e607de60 service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:14.832974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5d83e-74db-44ec-ad7b-b05728b5c295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:14.833640 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5d83e-74db-44ec-ad7b-b05728b5c295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:14.833921 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5d83e-74db-44ec-ad7b-b05728b5c295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:14.834229 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5d83e-74db-44ec-ad7b-b05728b5c295 admin admin] PATH_INFO: `/v3/users/391214020f734c8bb83e43c583c77965` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:14.835877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5d83e-74db-44ec-ad7b-b05728b5c295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=391214020f734c8bb83e43c583c77965)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:14.839976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5d83e-74db-44ec-ad7b-b05728b5c295 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:14.930653 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-40c5d83e-74db-44ec-ad7b-b05728b5c295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '391214020f734c8bb83e43c583c77965'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:14.949065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-40c5d83e-74db-44ec-ad7b-b05728b5c295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '391214020f734c8bb83e43c583c77965'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:14.964581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-40c5d83e-74db-44ec-ad7b-b05728b5c295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '391214020f734c8bb83e43c583c77965'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:14.973364 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-40c5d83e-74db-44ec-ad7b-b05728b5c295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '391214020f734c8bb83e43c583c77965'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:14.973808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-40c5d83e-74db-44ec-ad7b-b05728b5c295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '391214020f734c8bb83e43c583c77965'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:14.981739 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 929/1859] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:14 2022] DELETE /identity/v3/users/391214020f734c8bb83e43c583c77965 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:14.987894 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2a44b-a540-415e-9d0e-054cfbc56aa8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:14.988423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2a44b-a540-415e-9d0e-054cfbc56aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:14.988423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2a44b-a540-415e-9d0e-054cfbc56aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:15.009572 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-88a2a44b-a540-415e-9d0e-054cfbc56aa8 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:15.011070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-88a2a44b-a540-415e-9d0e-054cfbc56aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:15.106178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 931/1860] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:15.121202 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5d83e-74db-44ec-ad7b-b05728b5c295 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:15.123078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-40c5d83e-74db-44ec-ad7b-b05728b5c295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:15.206041 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5d83e-74db-44ec-ad7b-b05728b5c295 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:15.211146 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5faae-8cab-45c6-b179-a076a1accde2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:15.212136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5faae-8cab-45c6-b179-a076a1accde2 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:15.214344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5faae-8cab-45c6-b179-a076a1accde2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:15.214344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5faae-8cab-45c6-b179-a076a1accde2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:15.238665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5faae-8cab-45c6-b179-a076a1accde2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:15.240130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5faae-8cab-45c6-b179-a076a1accde2 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:15.251976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 930/1861] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:07:15.593202 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2a44b-a540-415e-9d0e-054cfbc56aa8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:15.594825 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-88a2a44b-a540-415e-9d0e-054cfbc56aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:15.608004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2a44b-a540-415e-9d0e-054cfbc56aa8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:15.615830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e849d-4dd7-4be6-a726-ed0ed800728e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:15.616816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e849d-4dd7-4be6-a726-ed0ed800728e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:15.617328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e849d-4dd7-4be6-a726-ed0ed800728e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:15.617731 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e849d-4dd7-4be6-a726-ed0ed800728e admin admin] PATH_INFO: `/v3/projects/80ec88ab4b5246b1aa8cecc9d2373d86` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:15.619112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e849d-4dd7-4be6-a726-ed0ed800728e admin admin] RBAC: Authorizing `identity:delete_project(project_id=80ec88ab4b5246b1aa8cecc9d2373d86)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:15.621756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e849d-4dd7-4be6-a726-ed0ed800728e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:15.691658 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ef3e849d-4dd7-4be6-a726-ed0ed800728e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80ec88ab4b5246b1aa8cecc9d2373d86', 'request_id': 'req-ef3e849d-4dd7-4be6-a726-ed0ed800728e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:15.709702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ef3e849d-4dd7-4be6-a726-ed0ed800728e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80ec88ab4b5246b1aa8cecc9d2373d86', 'request_id': 'req-ef3e849d-4dd7-4be6-a726-ed0ed800728e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:15.719465 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ef3e849d-4dd7-4be6-a726-ed0ed800728e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80ec88ab4b5246b1aa8cecc9d2373d86', 'request_id': 'req-ef3e849d-4dd7-4be6-a726-ed0ed800728e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:15.733921 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ef3e849d-4dd7-4be6-a726-ed0ed800728e admin admin] The token cache is being invalidate because project 80ec88ab4b5246b1aa8cecc9d2373d86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:15.734302 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ef3e849d-4dd7-4be6-a726-ed0ed800728e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:15.735026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ef3e849d-4dd7-4be6-a726-ed0ed800728e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:15.736888 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 932/1862] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:15 2022] DELETE /identity/v3/projects/80ec88ab4b5246b1aa8cecc9d2373d86 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:15.741254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5faae-8cab-45c6-b179-a076a1accde2 service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:15.753361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5faae-8cab-45c6-b179-a076a1accde2 service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:15.776343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:15.824801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2fb5a7-9a20-4d6e-b641-593a1388231c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:15.825463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2fb5a7-9a20-4d6e-b641-593a1388231c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:15.825786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2fb5a7-9a20-4d6e-b641-593a1388231c admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:15.826198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2fb5a7-9a20-4d6e-b641-593a1388231c admin admin] PATH_INFO: `/v3/users/d89cd45362b24150abfdeead616033ec` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:15.827433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2fb5a7-9a20-4d6e-b641-593a1388231c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d89cd45362b24150abfdeead616033ec)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:15.830158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2fb5a7-9a20-4d6e-b641-593a1388231c admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:15.924207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4b2fb5a7-9a20-4d6e-b641-593a1388231c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd89cd45362b24150abfdeead616033ec'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:15.939497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4b2fb5a7-9a20-4d6e-b641-593a1388231c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd89cd45362b24150abfdeead616033ec'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:15.963644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4b2fb5a7-9a20-4d6e-b641-593a1388231c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd89cd45362b24150abfdeead616033ec'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:15.983958 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4b2fb5a7-9a20-4d6e-b641-593a1388231c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd89cd45362b24150abfdeead616033ec'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:15.984749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4b2fb5a7-9a20-4d6e-b641-593a1388231c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd89cd45362b24150abfdeead616033ec'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:16.008122 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 931/1863] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:15 2022] DELETE /identity/v3/users/d89cd45362b24150abfdeead616033ec => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:16.623464 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e849d-4dd7-4be6-a726-ed0ed800728e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:16.624894 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ef3e849d-4dd7-4be6-a726-ed0ed800728e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:16.643797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e849d-4dd7-4be6-a726-ed0ed800728e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:16.726504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8b3a5-0fe0-46d1-8a99-85be603cd3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:16.727297 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8b3a5-0fe0-46d1-8a99-85be603cd3be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:16.727657 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8b3a5-0fe0-46d1-8a99-85be603cd3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:16.728005 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8b3a5-0fe0-46d1-8a99-85be603cd3be admin admin] PATH_INFO: `/v3/projects/64f7dde881d84ef6a302a6d57ecd6cfa` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:16.729570 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8b3a5-0fe0-46d1-8a99-85be603cd3be admin admin] RBAC: Authorizing `identity:delete_project(project_id=64f7dde881d84ef6a302a6d57ecd6cfa)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:16.731613 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8b3a5-0fe0-46d1-8a99-85be603cd3be admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:16.769748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:16.791425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d3f8b3a5-0fe0-46d1-8a99-85be603cd3be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64f7dde881d84ef6a302a6d57ecd6cfa', 'request_id': 'req-d3f8b3a5-0fe0-46d1-8a99-85be603cd3be'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:16.799064 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d3f8b3a5-0fe0-46d1-8a99-85be603cd3be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64f7dde881d84ef6a302a6d57ecd6cfa', 'request_id': 'req-d3f8b3a5-0fe0-46d1-8a99-85be603cd3be'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:16.805441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d3f8b3a5-0fe0-46d1-8a99-85be603cd3be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64f7dde881d84ef6a302a6d57ecd6cfa', 'request_id': 'req-d3f8b3a5-0fe0-46d1-8a99-85be603cd3be'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:16.816804 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d3f8b3a5-0fe0-46d1-8a99-85be603cd3be admin admin] The token cache is being invalidate because project 64f7dde881d84ef6a302a6d57ecd6cfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:16.817093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d3f8b3a5-0fe0-46d1-8a99-85be603cd3be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:16.817752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d3f8b3a5-0fe0-46d1-8a99-85be603cd3be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:16.819497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 933/1864] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:16 2022] DELETE /identity/v3/projects/64f7dde881d84ef6a302a6d57ecd6cfa => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:16.824196 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c271db-d91d-42fa-b07b-3e9e1d46f8d9 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:16.824712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c271db-d91d-42fa-b07b-3e9e1d46f8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:16.824812 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c271db-d91d-42fa-b07b-3e9e1d46f8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:16.850896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-c6c271db-d91d-42fa-b07b-3e9e1d46f8d9 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:16.852341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-c6c271db-d91d-42fa-b07b-3e9e1d46f8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:16.977127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 932/1865] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:07:16.987907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6a43a-191a-4436-a863-776315465db9 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:16.988195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6a43a-191a-4436-a863-776315465db9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:16.989125 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6a43a-191a-4436-a863-776315465db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:17.016010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-98b6a43a-191a-4436-a863-776315465db9 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:17.017582 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-98b6a43a-191a-4436-a863-776315465db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:17.027478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 934/1866] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:17.031280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c271db-d91d-42fa-b07b-3e9e1d46f8d9 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:17.042505 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c271db-d91d-42fa-b07b-3e9e1d46f8d9 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:17.048282 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5e934-6e85-49d8-88c4-c74a5361d2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:17.048924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5e934-6e85-49d8-88c4-c74a5361d2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:17.049205 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5e934-6e85-49d8-88c4-c74a5361d2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:17.049477 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5e934-6e85-49d8-88c4-c74a5361d2e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:17.050176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5e934-6e85-49d8-88c4-c74a5361d2e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:07:17.050470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5e934-6e85-49d8-88c4-c74a5361d2e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:17.052444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5e934-6e85-49d8-88c4-c74a5361d2e6 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:17.065646 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 933/1867] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:07:17.070096 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6a43a-191a-4436-a863-776315465db9 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:17.080225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6a43a-191a-4436-a863-776315465db9 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:17.084890 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff1263-99cc-4dc5-a447-7592802da926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:17.085497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff1263-99cc-4dc5-a447-7592802da926 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:17.085775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff1263-99cc-4dc5-a447-7592802da926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:17.086696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff1263-99cc-4dc5-a447-7592802da926 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:17.087343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff1263-99cc-4dc5-a447-7592802da926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:17.089214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff1263-99cc-4dc5-a447-7592802da926 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:17.116712 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 935/1868] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:07:17 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:17.121363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5e934-6e85-49d8-88c4-c74a5361d2e6 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:17.131521 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5e934-6e85-49d8-88c4-c74a5361d2e6 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:17.161184 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:17.209084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e2776-0325-44a7-b4b3-c1c89eeb74ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:17.209766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e2776-0325-44a7-b4b3-c1c89eeb74ff admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:17.210048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e2776-0325-44a7-b4b3-c1c89eeb74ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:17.210351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e2776-0325-44a7-b4b3-c1c89eeb74ff admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:17.211650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e2776-0325-44a7-b4b3-c1c89eeb74ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:17.214554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e2776-0325-44a7-b4b3-c1c89eeb74ff admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:17.243235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 934/1869] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:07:17 2022] POST /identity/v3/users => generated 290 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:17.252897 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff1263-99cc-4dc5-a447-7592802da926 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:17.265929 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff1263-99cc-4dc5-a447-7592802da926 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:17.271158 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32fb63-76d0-46b5-bbf9-9a613acfcd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:17.271779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32fb63-76d0-46b5-bbf9-9a613acfcd36 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:17.272057 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32fb63-76d0-46b5-bbf9-9a613acfcd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:17.272329 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32fb63-76d0-46b5-bbf9-9a613acfcd36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:17.272973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32fb63-76d0-46b5-bbf9-9a613acfcd36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:17.274713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32fb63-76d0-46b5-bbf9-9a613acfcd36 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:17.286775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 936/1870] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:17.291005 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e2776-0325-44a7-b4b3-c1c89eeb74ff admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:17.301449 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e2776-0325-44a7-b4b3-c1c89eeb74ff admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:17.306627 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76120e-cc8e-41fe-999d-c729862a5cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:17.307231 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76120e-cc8e-41fe-999d-c729862a5cae admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:17.307591 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76120e-cc8e-41fe-999d-c729862a5cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:17.307767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76120e-cc8e-41fe-999d-c729862a5cae admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:17.308630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76120e-cc8e-41fe-999d-c729862a5cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:17.310908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76120e-cc8e-41fe-999d-c729862a5cae admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:17.325836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 935/1871] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:17.326623 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32fb63-76d0-46b5-bbf9-9a613acfcd36 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:17.338502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32fb63-76d0-46b5-bbf9-9a613acfcd36 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:17.343571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb0fe9-91d3-4d30-b7cf-bb79d6215ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:17.344431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb0fe9-91d3-4d30-b7cf-bb79d6215ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:17.344784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb0fe9-91d3-4d30-b7cf-bb79d6215ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:17.345127 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb0fe9-91d3-4d30-b7cf-bb79d6215ec4 admin admin] PATH_INFO: `/v3/projects/56d9a95d2dc94d67953c8b87f374bc2a/users/5360b012d34d418a872f222b86c12d93/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:17.361407 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb0fe9-91d3-4d30-b7cf-bb79d6215ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56d9a95d2dc94d67953c8b87f374bc2a, user_id=5360b012d34d418a872f222b86c12d93, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:17.364531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb0fe9-91d3-4d30-b7cf-bb79d6215ec4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:17.382590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 937/1872] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:17 2022] PUT /identity/v3/projects/56d9a95d2dc94d67953c8b87f374bc2a/users/5360b012d34d418a872f222b86c12d93/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:07:17.392353 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76120e-cc8e-41fe-999d-c729862a5cae admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:17.394678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-8d76120e-cc8e-41fe-999d-c729862a5cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:17.395183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391eb81-81d4-4aa6-8f9f-3ea666149a81 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:17.395473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391eb81-81d4-4aa6-8f9f-3ea666149a81 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:17.395749 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391eb81-81d4-4aa6-8f9f-3ea666149a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:17.409705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76120e-cc8e-41fe-999d-c729862a5cae admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:17.429873 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c391eb81-81d4-4aa6-8f9f-3ea666149a81 None None] MFA Rules not processed for user `5360b012d34d418a872f222b86c12d93`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:17.431140 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c391eb81-81d4-4aa6-8f9f-3ea666149a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:17.508548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa377b-6296-4d9a-a021-7583b49342ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:17.509708 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa377b-6296-4d9a-a021-7583b49342ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:17.510537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa377b-6296-4d9a-a021-7583b49342ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:17.510537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa377b-6296-4d9a-a021-7583b49342ff admin admin] PATH_INFO: `/v3/users/b0effeb7060e43d88390e4f49fbc8c0f` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:17.517441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa377b-6296-4d9a-a021-7583b49342ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0effeb7060e43d88390e4f49fbc8c0f)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:17.519746 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa377b-6296-4d9a-a021-7583b49342ff admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:17.523373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 938/1873] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:17.540326 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391eb81-81d4-4aa6-8f9f-3ea666149a81 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:17.592042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391eb81-81d4-4aa6-8f9f-3ea666149a81 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:17.603709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bcb09-9672-4695-9085-105f1618862c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:17.604347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bcb09-9672-4695-9085-105f1618862c admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:17.604623 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bcb09-9672-4695-9085-105f1618862c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:17.604895 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bcb09-9672-4695-9085-105f1618862c admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:17.609947 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007bcb09-9672-4695-9085-105f1618862c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:17.615213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007bcb09-9672-4695-9085-105f1618862c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:17.674351 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 939/1874] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:07:17 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:17.674351 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bcb09-9672-4695-9085-105f1618862c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:17.693009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:17.712989 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-e7aa377b-6296-4d9a-a021-7583b49342ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0effeb7060e43d88390e4f49fbc8c0f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:17.730970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-e7aa377b-6296-4d9a-a021-7583b49342ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0effeb7060e43d88390e4f49fbc8c0f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:17.766701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-e7aa377b-6296-4d9a-a021-7583b49342ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0effeb7060e43d88390e4f49fbc8c0f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:17.786342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-e7aa377b-6296-4d9a-a021-7583b49342ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0effeb7060e43d88390e4f49fbc8c0f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:17.787409 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-e7aa377b-6296-4d9a-a021-7583b49342ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0effeb7060e43d88390e4f49fbc8c0f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:17.796931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bcb09-9672-4695-9085-105f1618862c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:17.802769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 936/1875] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:17 2022] DELETE /identity/v3/users/b0effeb7060e43d88390e4f49fbc8c0f => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:17.807440 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a951a10-34ef-4edc-9a22-bc67e7d3985f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:17.807728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a951a10-34ef-4edc-9a22-bc67e7d3985f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:17.808006 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a951a10-34ef-4edc-9a22-bc67e7d3985f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:17.875963 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-2a951a10-34ef-4edc-9a22-bc67e7d3985f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:17.877785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2a951a10-34ef-4edc-9a22-bc67e7d3985f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:17.932326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:17.968138 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb28240-d28c-427d-bbce-c75df62fb587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:17.969042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb28240-d28c-427d-bbce-c75df62fb587 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:17.969186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb28240-d28c-427d-bbce-c75df62fb587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:17.969938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb28240-d28c-427d-bbce-c75df62fb587 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:17.970794 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb28240-d28c-427d-bbce-c75df62fb587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:17.973746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb28240-d28c-427d-bbce-c75df62fb587 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:18.013286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 940/1876] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:07:17 2022] POST /identity/v3/users => generated 297 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:07:18.014461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb28240-d28c-427d-bbce-c75df62fb587 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:18.017261 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bcb28240-d28c-427d-bbce-c75df62fb587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:18.034937 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb28240-d28c-427d-bbce-c75df62fb587 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:18.035245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 937/1877] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:07:18.040419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082c34a-9260-4f04-ad18-f0bc3dd6bbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:18.041291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082c34a-9260-4f04-ad18-f0bc3dd6bbed admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:18.041635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082c34a-9260-4f04-ad18-f0bc3dd6bbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:18.041952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082c34a-9260-4f04-ad18-f0bc3dd6bbed admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:18.042728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082c34a-9260-4f04-ad18-f0bc3dd6bbed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:18.044626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082c34a-9260-4f04-ad18-f0bc3dd6bbed admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:18.046594 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a951a10-34ef-4edc-9a22-bc67e7d3985f None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:18.048950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2a951a10-34ef-4edc-9a22-bc67e7d3985f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:18.057814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 941/1878] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:18.062882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082c34a-9260-4f04-ad18-f0bc3dd6bbed admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:18.074660 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082c34a-9260-4f04-ad18-f0bc3dd6bbed admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:18.079610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8b8d0-3743-400f-a64b-43da4c427c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:18.080288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8b8d0-3743-400f-a64b-43da4c427c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:18.081214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8b8d0-3743-400f-a64b-43da4c427c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:18.081491 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8b8d0-3743-400f-a64b-43da4c427c1e admin admin] PATH_INFO: `/v3/projects/f12dc53b909b46bfb1c882b3f86722c7/users/164705b7614a42cb81e4284512f3832a/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:18.102084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8b8d0-3743-400f-a64b-43da4c427c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f12dc53b909b46bfb1c882b3f86722c7, user_id=164705b7614a42cb81e4284512f3832a, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:18.105310 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8b8d0-3743-400f-a64b-43da4c427c1e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:18.128475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 942/1879] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:18 2022] PUT /identity/v3/projects/f12dc53b909b46bfb1c882b3f86722c7/users/164705b7614a42cb81e4284512f3832a/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:18.130199 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8b8d0-3743-400f-a64b-43da4c427c1e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:18.135461 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a951a10-34ef-4edc-9a22-bc67e7d3985f None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:18.208342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b1c1a-1a28-4141-afb1-60c081b271d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:18.208966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b1c1a-1a28-4141-afb1-60c081b271d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:18.209309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b1c1a-1a28-4141-afb1-60c081b271d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:18.209724 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b1c1a-1a28-4141-afb1-60c081b271d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:18.217111 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8b8d0-3743-400f-a64b-43da4c427c1e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:18.227006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075c2dd-9788-4e33-b8c6-bc23b4dbb1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:18.227957 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075c2dd-9788-4e33-b8c6-bc23b4dbb1dc admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:18.228051 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075c2dd-9788-4e33-b8c6-bc23b4dbb1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:18.228344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075c2dd-9788-4e33-b8c6-bc23b4dbb1dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:18.229487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075c2dd-9788-4e33-b8c6-bc23b4dbb1dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:18.230348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b1c1a-1a28-4141-afb1-60c081b271d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:18.231187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075c2dd-9788-4e33-b8c6-bc23b4dbb1dc admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:18.231929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b1c1a-1a28-4141-afb1-60c081b271d5 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:18.248436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 943/1880] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:18.248669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 938/1881] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:07:18.249867 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075c2dd-9788-4e33-b8c6-bc23b4dbb1dc admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:18.263052 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075c2dd-9788-4e33-b8c6-bc23b4dbb1dc admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:18.268920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741d406-5c6d-4c97-b73d-c54c8d41c27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:18.269536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741d406-5c6d-4c97-b73d-c54c8d41c27f admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:18.269815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741d406-5c6d-4c97-b73d-c54c8d41c27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:18.270177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741d406-5c6d-4c97-b73d-c54c8d41c27f admin admin] PATH_INFO: `/v3/projects/f12dc53b909b46bfb1c882b3f86722c7/users/164705b7614a42cb81e4284512f3832a/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:18.272958 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741d406-5c6d-4c97-b73d-c54c8d41c27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f12dc53b909b46bfb1c882b3f86722c7, user_id=164705b7614a42cb81e4284512f3832a, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:18.278629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741d406-5c6d-4c97-b73d-c54c8d41c27f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:18.300275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 944/1882] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:18 2022] PUT /identity/v3/projects/f12dc53b909b46bfb1c882b3f86722c7/users/164705b7614a42cb81e4284512f3832a/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:18.316836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357e5b5-931d-42ac-ae25-8d616f40ca80 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:18.318406 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357e5b5-931d-42ac-ae25-8d616f40ca80 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:18.319925 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357e5b5-931d-42ac-ae25-8d616f40ca80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:18.365267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-7357e5b5-931d-42ac-ae25-8d616f40ca80 None None] MFA Rules not processed for user `164705b7614a42cb81e4284512f3832a`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:18.366539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7357e5b5-931d-42ac-ae25-8d616f40ca80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:18.496092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 939/1883] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:07:18.504550 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741d406-5c6d-4c97-b73d-c54c8d41c27f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:18.508914 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a741d406-5c6d-4c97-b73d-c54c8d41c27f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:18.520293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:18.606364 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741d406-5c6d-4c97-b73d-c54c8d41c27f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:18.626227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33322f9f-d7d9-4f2e-9e05-78d1bed6f7de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:18.627032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33322f9f-d7d9-4f2e-9e05-78d1bed6f7de service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:18.627316 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33322f9f-d7d9-4f2e-9e05-78d1bed6f7de service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:18.627608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33322f9f-d7d9-4f2e-9e05-78d1bed6f7de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:18.652480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33322f9f-d7d9-4f2e-9e05-78d1bed6f7de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:18.657650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33322f9f-d7d9-4f2e-9e05-78d1bed6f7de service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:18.669507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 945/1884] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:18.779350 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357e5b5-931d-42ac-ae25-8d616f40ca80 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:18.783673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7357e5b5-931d-42ac-ae25-8d616f40ca80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:18.817386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357e5b5-931d-42ac-ae25-8d616f40ca80 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:18.822629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33322f9f-d7d9-4f2e-9e05-78d1bed6f7de service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:18.824216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-33322f9f-d7d9-4f2e-9e05-78d1bed6f7de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:18.828162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d34a4c-51dc-4a6f-a61a-31c951c32d08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:18.829120 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d34a4c-51dc-4a6f-a61a-31c951c32d08 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:18.829647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d34a4c-51dc-4a6f-a61a-31c951c32d08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:18.830178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d34a4c-51dc-4a6f-a61a-31c951c32d08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:18.832318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b6d34a4c-51dc-4a6f-a61a-31c951c32d08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:18.845087 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:18.981751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d34a4c-51dc-4a6f-a61a-31c951c32d08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:18.983238 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d34a4c-51dc-4a6f-a61a-31c951c32d08 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:18.997537 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33322f9f-d7d9-4f2e-9e05-78d1bed6f7de service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:18.998252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 940/1885] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:19.005063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a2291-114f-448e-9bec-e665ca2581bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:19.007711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a2291-114f-448e-9bec-e665ca2581bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:19.008794 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a2291-114f-448e-9bec-e665ca2581bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:19.009301 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a2291-114f-448e-9bec-e665ca2581bd admin admin] PATH_INFO: `/v3/projects/a1092f5d05ee45d88eac9697f1950e5f` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:19.011407 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a2291-114f-448e-9bec-e665ca2581bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1092f5d05ee45d88eac9697f1950e5f)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:19.013988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a2291-114f-448e-9bec-e665ca2581bd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:19.104852 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6c1a2291-114f-448e-9bec-e665ca2581bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1092f5d05ee45d88eac9697f1950e5f', 'request_id': 'req-6c1a2291-114f-448e-9bec-e665ca2581bd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:19.119514 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6c1a2291-114f-448e-9bec-e665ca2581bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1092f5d05ee45d88eac9697f1950e5f', 'request_id': 'req-6c1a2291-114f-448e-9bec-e665ca2581bd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:19.128351 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6c1a2291-114f-448e-9bec-e665ca2581bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1092f5d05ee45d88eac9697f1950e5f', 'request_id': 'req-6c1a2291-114f-448e-9bec-e665ca2581bd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:19.161614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6c1a2291-114f-448e-9bec-e665ca2581bd admin admin] The token cache is being invalidate because project a1092f5d05ee45d88eac9697f1950e5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:19.162558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6c1a2291-114f-448e-9bec-e665ca2581bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:19.165529 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6c1a2291-114f-448e-9bec-e665ca2581bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:19.170633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 946/1886] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:18 2022] DELETE /identity/v3/projects/a1092f5d05ee45d88eac9697f1950e5f => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:07:19.175354 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d34a4c-51dc-4a6f-a61a-31c951c32d08 service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:19.190729 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d34a4c-51dc-4a6f-a61a-31c951c32d08 service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:19.260109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c08c7-f3ec-47c4-b912-23849470aa02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:19.260775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c08c7-f3ec-47c4-b912-23849470aa02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:19.261049 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c08c7-f3ec-47c4-b912-23849470aa02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:19.261360 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c08c7-f3ec-47c4-b912-23849470aa02 admin admin] PATH_INFO: `/v3/users/c90b8a32a236468d9ca58260d0d6b94d` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:19.262536 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c08c7-f3ec-47c4-b912-23849470aa02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c90b8a32a236468d9ca58260d0d6b94d)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:19.264472 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c08c7-f3ec-47c4-b912-23849470aa02 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:19.310416 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b46c08c7-f3ec-47c4-b912-23849470aa02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c90b8a32a236468d9ca58260d0d6b94d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:19.318423 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b46c08c7-f3ec-47c4-b912-23849470aa02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c90b8a32a236468d9ca58260d0d6b94d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:19.330786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b46c08c7-f3ec-47c4-b912-23849470aa02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c90b8a32a236468d9ca58260d0d6b94d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:19.337605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b46c08c7-f3ec-47c4-b912-23849470aa02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c90b8a32a236468d9ca58260d0d6b94d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:19.338286 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b46c08c7-f3ec-47c4-b912-23849470aa02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c90b8a32a236468d9ca58260d0d6b94d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:19.344054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 941/1887] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:19 2022] DELETE /identity/v3/users/c90b8a32a236468d9ca58260d0d6b94d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:19.673366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a2291-114f-448e-9bec-e665ca2581bd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:19.675001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6c1a2291-114f-448e-9bec-e665ca2581bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:19.688824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a2291-114f-448e-9bec-e665ca2581bd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:19.743228 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:19.775094 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4105a-1e9b-4652-9320-ec71b632fb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:19.776016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4105a-1e9b-4652-9320-ec71b632fb2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:19.776016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4105a-1e9b-4652-9320-ec71b632fb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:19.776315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4105a-1e9b-4652-9320-ec71b632fb2e admin admin] PATH_INFO: `/v3/projects/3dfa9442af404ff681936b42e0c86f15` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:19.777721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4105a-1e9b-4652-9320-ec71b632fb2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dfa9442af404ff681936b42e0c86f15)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:19.779819 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4105a-1e9b-4652-9320-ec71b632fb2e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:19.844851 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-53a4105a-1e9b-4652-9320-ec71b632fb2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dfa9442af404ff681936b42e0c86f15', 'request_id': 'req-53a4105a-1e9b-4652-9320-ec71b632fb2e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:19.870874 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-53a4105a-1e9b-4652-9320-ec71b632fb2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dfa9442af404ff681936b42e0c86f15', 'request_id': 'req-53a4105a-1e9b-4652-9320-ec71b632fb2e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:19.878595 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-53a4105a-1e9b-4652-9320-ec71b632fb2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dfa9442af404ff681936b42e0c86f15', 'request_id': 'req-53a4105a-1e9b-4652-9320-ec71b632fb2e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:19.893706 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-53a4105a-1e9b-4652-9320-ec71b632fb2e admin admin] The token cache is being invalidate because project 3dfa9442af404ff681936b42e0c86f15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:19.894313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-53a4105a-1e9b-4652-9320-ec71b632fb2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:19.895205 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-53a4105a-1e9b-4652-9320-ec71b632fb2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:19.897394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 947/1888] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:19 2022] DELETE /identity/v3/projects/3dfa9442af404ff681936b42e0c86f15 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:19.902416 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f70f38-78e4-470d-952d-b35814cd5458 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:19.903156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f70f38-78e4-470d-952d-b35814cd5458 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:19.903464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f70f38-78e4-470d-952d-b35814cd5458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:19.932918 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d9f70f38-78e4-470d-952d-b35814cd5458 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:19.934197 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d9f70f38-78e4-470d-952d-b35814cd5458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:19.981273 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:20.039689 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 942/1889] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:20.047422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b18a5e-7abe-4cde-85a9-e561803e5df8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:20.047707 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b18a5e-7abe-4cde-85a9-e561803e5df8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:20.047985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b18a5e-7abe-4cde-85a9-e561803e5df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:20.072006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-03b18a5e-7abe-4cde-85a9-e561803e5df8 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:20.073925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-03b18a5e-7abe-4cde-85a9-e561803e5df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:20.084237 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 948/1890] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:20.088624 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f70f38-78e4-470d-952d-b35814cd5458 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:20.099884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f70f38-78e4-470d-952d-b35814cd5458 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:20.105517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c215f96-82e3-4c83-86fb-e4eb04401d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:20.106160 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c215f96-82e3-4c83-86fb-e4eb04401d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:20.106458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c215f96-82e3-4c83-86fb-e4eb04401d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:20.106723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c215f96-82e3-4c83-86fb-e4eb04401d65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:20.107383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c215f96-82e3-4c83-86fb-e4eb04401d65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:07:20.107682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c215f96-82e3-4c83-86fb-e4eb04401d65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:20.109273 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c215f96-82e3-4c83-86fb-e4eb04401d65 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:20.126005 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 943/1891] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:07:20.127457 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b18a5e-7abe-4cde-85a9-e561803e5df8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:20.137358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b18a5e-7abe-4cde-85a9-e561803e5df8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:20.142066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2cef9-af61-4ec8-ac91-2393114bd3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:20.142720 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2cef9-af61-4ec8-ac91-2393114bd3bd admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:20.142996 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2cef9-af61-4ec8-ac91-2393114bd3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:20.143269 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2cef9-af61-4ec8-ac91-2393114bd3bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:20.143917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd2cef9-af61-4ec8-ac91-2393114bd3bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:20.145807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd2cef9-af61-4ec8-ac91-2393114bd3bd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:20.175139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 949/1892] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:07:20 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:20.179341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c215f96-82e3-4c83-86fb-e4eb04401d65 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:20.196633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c215f96-82e3-4c83-86fb-e4eb04401d65 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:20.321964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b6ca3-f63a-4b35-bd59-a637d3f13af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:20.322620 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b6ca3-f63a-4b35-bd59-a637d3f13af0 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:20.322937 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b6ca3-f63a-4b35-bd59-a637d3f13af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:20.323347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b6ca3-f63a-4b35-bd59-a637d3f13af0 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:20.323883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b6ca3-f63a-4b35-bd59-a637d3f13af0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:20.325847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b6ca3-f63a-4b35-bd59-a637d3f13af0 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:20.352844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 944/1893] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:07:20 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:20.358420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2cef9-af61-4ec8-ac91-2393114bd3bd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:20.370876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2cef9-af61-4ec8-ac91-2393114bd3bd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:20.376088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd97cde-9c55-4e56-9183-5a7c9224227c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:20.376870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97cde-9c55-4e56-9183-5a7c9224227c admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:20.377102 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97cde-9c55-4e56-9183-5a7c9224227c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:20.377372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97cde-9c55-4e56-9183-5a7c9224227c admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:20.378245 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd97cde-9c55-4e56-9183-5a7c9224227c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:20.379839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd97cde-9c55-4e56-9183-5a7c9224227c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:20.390796 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 950/1894] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:20.395413 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b6ca3-f63a-4b35-bd59-a637d3f13af0 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:20.409062 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b6ca3-f63a-4b35-bd59-a637d3f13af0 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:20.415922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7bd9b-fd2f-425a-88de-9261e096dbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:20.417215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7bd9b-fd2f-425a-88de-9261e096dbca admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:20.417215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7bd9b-fd2f-425a-88de-9261e096dbca admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:20.417407 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7bd9b-fd2f-425a-88de-9261e096dbca admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:20.417954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7bd9b-fd2f-425a-88de-9261e096dbca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:20.419766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7bd9b-fd2f-425a-88de-9261e096dbca admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:20.431585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 945/1895] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:20.436197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd97cde-9c55-4e56-9183-5a7c9224227c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:20.446739 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd97cde-9c55-4e56-9183-5a7c9224227c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:20.451647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946da881-38b0-4e50-8b42-2f18eaadefcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:20.452384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946da881-38b0-4e50-8b42-2f18eaadefcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:20.452737 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946da881-38b0-4e50-8b42-2f18eaadefcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:20.453083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946da881-38b0-4e50-8b42-2f18eaadefcd admin admin] PATH_INFO: `/v3/projects/f5536985a156470cbb865649fd1abfbd/users/cead08c2f8ed4b918419804d705f8fe0/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:20.475623 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946da881-38b0-4e50-8b42-2f18eaadefcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5536985a156470cbb865649fd1abfbd, user_id=cead08c2f8ed4b918419804d705f8fe0, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:20.479962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946da881-38b0-4e50-8b42-2f18eaadefcd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:20.510032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 951/1896] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:20 2022] PUT /identity/v3/projects/f5536985a156470cbb865649fd1abfbd/users/cead08c2f8ed4b918419804d705f8fe0/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:20.522243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbebe3-60a9-4b87-baef-1a4a22d48737 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:20.522782 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbebe3-60a9-4b87-baef-1a4a22d48737 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:20.523272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbebe3-60a9-4b87-baef-1a4a22d48737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:20.574700 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-fbcbebe3-60a9-4b87-baef-1a4a22d48737 None None] MFA Rules not processed for user `cead08c2f8ed4b918419804d705f8fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:20.576067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-fbcbebe3-60a9-4b87-baef-1a4a22d48737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:20.656439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 946/1897] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:20.661104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946da881-38b0-4e50-8b42-2f18eaadefcd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:20.729101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946da881-38b0-4e50-8b42-2f18eaadefcd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:20.733827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb0d0a-f44b-4e57-bd55-2d31a3167a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:20.734479 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb0d0a-f44b-4e57-bd55-2d31a3167a18 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:20.734758 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb0d0a-f44b-4e57-bd55-2d31a3167a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:20.735029 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb0d0a-f44b-4e57-bd55-2d31a3167a18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:20.735701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb0d0a-f44b-4e57-bd55-2d31a3167a18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:20.737607 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb0d0a-f44b-4e57-bd55-2d31a3167a18 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:20.743764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:20.768353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 952/1898] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:07:20 2022] POST /identity/v3/projects => generated 391 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:20.772659 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcbebe3-60a9-4b87-baef-1a4a22d48737 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:20.784002 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcbebe3-60a9-4b87-baef-1a4a22d48737 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:20.805420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:20.852563 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbaaba4-5565-41b6-9759-5f478d8a344e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:20.854221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbaaba4-5565-41b6-9759-5f478d8a344e admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:20.854586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbaaba4-5565-41b6-9759-5f478d8a344e admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:20.854905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbaaba4-5565-41b6-9759-5f478d8a344e admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:20.855637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbaaba4-5565-41b6-9759-5f478d8a344e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:20.858061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbaaba4-5565-41b6-9759-5f478d8a344e admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:20.908216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 947/1899] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:07:20 2022] POST /identity/v3/users => generated 302 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:20.910052 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb0d0a-f44b-4e57-bd55-2d31a3167a18 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:20.924483 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb0d0a-f44b-4e57-bd55-2d31a3167a18 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:20.933085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61abfd-2a6e-4a04-96ae-e9b9d0324b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:20.934005 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61abfd-2a6e-4a04-96ae-e9b9d0324b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:20.934472 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61abfd-2a6e-4a04-96ae-e9b9d0324b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:20.935531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61abfd-2a6e-4a04-96ae-e9b9d0324b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:20.937130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61abfd-2a6e-4a04-96ae-e9b9d0324b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:20.939505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61abfd-2a6e-4a04-96ae-e9b9d0324b18 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:20.952564 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 953/1900] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:20.957026 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbaaba4-5565-41b6-9759-5f478d8a344e admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:20.968251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbaaba4-5565-41b6-9759-5f478d8a344e admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:20.974100 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb5217-ebf3-4ef2-b2b1-bf84456b7f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:20.975133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb5217-ebf3-4ef2-b2b1-bf84456b7f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:20.975554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb5217-ebf3-4ef2-b2b1-bf84456b7f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:20.976985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb5217-ebf3-4ef2-b2b1-bf84456b7f38 admin admin] PATH_INFO: `/v3/projects/59fff47f3e034672b3e0e523558d9933/users/c4f5885898f6420d83d8558af7343114/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:20.999074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb5217-ebf3-4ef2-b2b1-bf84456b7f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59fff47f3e034672b3e0e523558d9933, user_id=c4f5885898f6420d83d8558af7343114, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:21.002536 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb5217-ebf3-4ef2-b2b1-bf84456b7f38 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:21.017877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 948/1901] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:20 2022] PUT /identity/v3/projects/59fff47f3e034672b3e0e523558d9933/users/c4f5885898f6420d83d8558af7343114/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:07:21.021750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61abfd-2a6e-4a04-96ae-e9b9d0324b18 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:21.039971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61abfd-2a6e-4a04-96ae-e9b9d0324b18 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:21.171236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44938d-6fba-4e65-9b59-e66341f1f6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:21.172065 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44938d-6fba-4e65-9b59-e66341f1f6c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:21.173073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44938d-6fba-4e65-9b59-e66341f1f6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:21.173073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44938d-6fba-4e65-9b59-e66341f1f6c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:21.173290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44938d-6fba-4e65-9b59-e66341f1f6c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:21.175950 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44938d-6fba-4e65-9b59-e66341f1f6c5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:21.192334 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 954/1902] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:21.196554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb5217-ebf3-4ef2-b2b1-bf84456b7f38 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:21.208542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb5217-ebf3-4ef2-b2b1-bf84456b7f38 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:21.214792 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873da28-dea3-4714-bf36-fc841f90e297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:21.215761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873da28-dea3-4714-bf36-fc841f90e297 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:21.216187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873da28-dea3-4714-bf36-fc841f90e297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:21.217172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873da28-dea3-4714-bf36-fc841f90e297 admin admin] PATH_INFO: `/v3/projects/59fff47f3e034672b3e0e523558d9933/users/c4f5885898f6420d83d8558af7343114/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:21.219445 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873da28-dea3-4714-bf36-fc841f90e297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59fff47f3e034672b3e0e523558d9933, user_id=c4f5885898f6420d83d8558af7343114, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:21.223297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873da28-dea3-4714-bf36-fc841f90e297 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:21.256235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688aa10-7062-48c2-8c41-15bf8709da71 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:21.256771 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688aa10-7062-48c2-8c41-15bf8709da71 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:21.257864 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688aa10-7062-48c2-8c41-15bf8709da71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:21.261724 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 949/1903] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:21 2022] PUT /identity/v3/projects/59fff47f3e034672b3e0e523558d9933/users/c4f5885898f6420d83d8558af7343114/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:21.305745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-6688aa10-7062-48c2-8c41-15bf8709da71 None None] MFA Rules not processed for user `c4f5885898f6420d83d8558af7343114`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:21.308018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6688aa10-7062-48c2-8c41-15bf8709da71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:21.456298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 955/1904] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:21 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:21.464888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873da28-dea3-4714-bf36-fc841f90e297 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:21.469921 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7873da28-dea3-4714-bf36-fc841f90e297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:21.567869 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873da28-dea3-4714-bf36-fc841f90e297 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:21.576492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f8bea-3922-4405-9555-47b0757c60e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:21.577130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f8bea-3922-4405-9555-47b0757c60e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:21.577406 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f8bea-3922-4405-9555-47b0757c60e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:21.577675 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f8bea-3922-4405-9555-47b0757c60e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:21.591205 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f8bea-3922-4405-9555-47b0757c60e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:21.609690 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f8bea-3922-4405-9555-47b0757c60e3 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:21.643545 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 950/1905] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:21 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:21.741502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688aa10-7062-48c2-8c41-15bf8709da71 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:21.756243 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688aa10-7062-48c2-8c41-15bf8709da71 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:21.760981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5606372-27da-417d-b083-bf7c7c596b2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:21.762794 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5606372-27da-417d-b083-bf7c7c596b2f service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:21.763220 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5606372-27da-417d-b083-bf7c7c596b2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:21.763682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5606372-27da-417d-b083-bf7c7c596b2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:21.910944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:21.919043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5606372-27da-417d-b083-bf7c7c596b2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:21.920877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5606372-27da-417d-b083-bf7c7c596b2f service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:21.960545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 956/1906] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:21 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:24.499597 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde24997-0588-41be-980e-731e717a098a None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:24.500162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde24997-0588-41be-980e-731e717a098a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:24.500162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde24997-0588-41be-980e-731e717a098a None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:24.501542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 951/1907] 10.222.0.62 () {54 vars in 921 bytes} [Fri Sep 9 12:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:24.511358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5b883-847a-46b4-8c1d-90e57f218c04 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:24.511831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5b883-847a-46b4-8c1d-90e57f218c04 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:24.511949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5b883-847a-46b4-8c1d-90e57f218c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:24.531688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-97f5b883-847a-46b4-8c1d-90e57f218c04 None None] MFA Rules not processed for user `c58bae3ef7784671ae247f4ad4199c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:24.532914 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-97f5b883-847a-46b4-8c1d-90e57f218c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:24.608936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 957/1908] 10.222.0.62 () {58 vars in 1018 bytes} [Fri Sep 9 12:07:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:24.622601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde24997-0588-41be-980e-731e717a098a None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:24.624473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-dde24997-0588-41be-980e-731e717a098a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:24.641728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde24997-0588-41be-980e-731e717a098a None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:24.649813 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736f11e-70a0-4fc0-9349-43e517312494 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:24.650540 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736f11e-70a0-4fc0-9349-43e517312494 service placement] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:24.650940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736f11e-70a0-4fc0-9349-43e517312494 service placement] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:24.651254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736f11e-70a0-4fc0-9349-43e517312494 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:24.660990 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9736f11e-70a0-4fc0-9349-43e517312494 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:24.663239 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9736f11e-70a0-4fc0-9349-43e517312494 service placement] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:24.674011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 952/1909] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:25.138485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5b883-847a-46b4-8c1d-90e57f218c04 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:25.139935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-97f5b883-847a-46b4-8c1d-90e57f218c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:25.206613 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5b883-847a-46b4-8c1d-90e57f218c04 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:25.216497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11051f47-5588-46ca-97fa-5cb88888997d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:25.217180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11051f47-5588-46ca-97fa-5cb88888997d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:25.217461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11051f47-5588-46ca-97fa-5cb88888997d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:25.217735 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11051f47-5588-46ca-97fa-5cb88888997d admin admin] PATH_INFO: `/v3/users/095270e399c64de1911733ff2940cfff` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:25.218916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11051f47-5588-46ca-97fa-5cb88888997d admin admin] RBAC: Authorizing `identity:delete_user(user_id=095270e399c64de1911733ff2940cfff)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:25.220840 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11051f47-5588-46ca-97fa-5cb88888997d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:25.271539 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-11051f47-5588-46ca-97fa-5cb88888997d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '095270e399c64de1911733ff2940cfff'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:25.272329 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-11051f47-5588-46ca-97fa-5cb88888997d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '095270e399c64de1911733ff2940cfff'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:25.280109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-11051f47-5588-46ca-97fa-5cb88888997d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '095270e399c64de1911733ff2940cfff'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:25.288255 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-11051f47-5588-46ca-97fa-5cb88888997d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '095270e399c64de1911733ff2940cfff'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:25.301347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-11051f47-5588-46ca-97fa-5cb88888997d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '095270e399c64de1911733ff2940cfff'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:25.307748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 958/1910] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:25 2022] DELETE /identity/v3/users/095270e399c64de1911733ff2940cfff => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:25.483935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736f11e-70a0-4fc0-9349-43e517312494 service placement] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:25.501185 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736f11e-70a0-4fc0-9349-43e517312494 service placement] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:25.563931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71ab2e-4541-4149-bfe4-7347423f1d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:25.564576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71ab2e-4541-4149-bfe4-7347423f1d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:25.564854 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71ab2e-4541-4149-bfe4-7347423f1d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:25.565146 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71ab2e-4541-4149-bfe4-7347423f1d95 admin admin] PATH_INFO: `/v3/projects/3458ea8e53454989a11a0eebd9d26e66` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:25.566469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71ab2e-4541-4149-bfe4-7347423f1d95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3458ea8e53454989a11a0eebd9d26e66)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:25.568347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71ab2e-4541-4149-bfe4-7347423f1d95 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:25.577104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:25.620237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-dd71ab2e-4541-4149-bfe4-7347423f1d95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3458ea8e53454989a11a0eebd9d26e66', 'request_id': 'req-dd71ab2e-4541-4149-bfe4-7347423f1d95'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:25.627469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-dd71ab2e-4541-4149-bfe4-7347423f1d95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3458ea8e53454989a11a0eebd9d26e66', 'request_id': 'req-dd71ab2e-4541-4149-bfe4-7347423f1d95'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:25.633092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-dd71ab2e-4541-4149-bfe4-7347423f1d95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3458ea8e53454989a11a0eebd9d26e66', 'request_id': 'req-dd71ab2e-4541-4149-bfe4-7347423f1d95'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:25.644520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-dd71ab2e-4541-4149-bfe4-7347423f1d95 admin admin] The token cache is being invalidate because project 3458ea8e53454989a11a0eebd9d26e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:25.644796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-dd71ab2e-4541-4149-bfe4-7347423f1d95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:25.645684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-dd71ab2e-4541-4149-bfe4-7347423f1d95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:25.647473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 953/1911] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:25 2022] DELETE /identity/v3/projects/3458ea8e53454989a11a0eebd9d26e66 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:25.652078 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11051f47-5588-46ca-97fa-5cb88888997d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:25.653512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-11051f47-5588-46ca-97fa-5cb88888997d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:25.667362 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11051f47-5588-46ca-97fa-5cb88888997d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:25.676052 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:25.726047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddccaba-ee88-43f0-9f69-fc268ee2dffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:25.727331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddccaba-ee88-43f0-9f69-fc268ee2dffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:25.727606 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddccaba-ee88-43f0-9f69-fc268ee2dffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:25.727877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddccaba-ee88-43f0-9f69-fc268ee2dffa admin admin] PATH_INFO: `/v3/users/6e638bbbe18f432392f34a4cd33edbd7` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:25.729002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddccaba-ee88-43f0-9f69-fc268ee2dffa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e638bbbe18f432392f34a4cd33edbd7)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:25.730966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddccaba-ee88-43f0-9f69-fc268ee2dffa admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:25.779778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fddccaba-ee88-43f0-9f69-fc268ee2dffa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e638bbbe18f432392f34a4cd33edbd7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:25.780429 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fddccaba-ee88-43f0-9f69-fc268ee2dffa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e638bbbe18f432392f34a4cd33edbd7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:25.787419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fddccaba-ee88-43f0-9f69-fc268ee2dffa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e638bbbe18f432392f34a4cd33edbd7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:25.795375 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fddccaba-ee88-43f0-9f69-fc268ee2dffa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e638bbbe18f432392f34a4cd33edbd7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:25.807947 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fddccaba-ee88-43f0-9f69-fc268ee2dffa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e638bbbe18f432392f34a4cd33edbd7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:25.814374 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 959/1912] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:25 2022] DELETE /identity/v3/users/6e638bbbe18f432392f34a4cd33edbd7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:07:26.002208 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71ab2e-4541-4149-bfe4-7347423f1d95 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:26.003821 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-dd71ab2e-4541-4149-bfe4-7347423f1d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:26.022763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71ab2e-4541-4149-bfe4-7347423f1d95 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:26.107688 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6d176-9436-4ef7-a921-e518b686e57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:26.108391 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6d176-9436-4ef7-a921-e518b686e57e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:26.108668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6d176-9436-4ef7-a921-e518b686e57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:26.108947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6d176-9436-4ef7-a921-e518b686e57e admin admin] PATH_INFO: `/v3/projects/5eccee94872840afacff642b7d05cb72` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:26.110098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6d176-9436-4ef7-a921-e518b686e57e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eccee94872840afacff642b7d05cb72)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:26.112034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6d176-9436-4ef7-a921-e518b686e57e admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:26.165264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f3f6d176-9436-4ef7-a921-e518b686e57e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eccee94872840afacff642b7d05cb72', 'request_id': 'req-f3f6d176-9436-4ef7-a921-e518b686e57e'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:26.174085 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f3f6d176-9436-4ef7-a921-e518b686e57e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eccee94872840afacff642b7d05cb72', 'request_id': 'req-f3f6d176-9436-4ef7-a921-e518b686e57e'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:26.180338 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f3f6d176-9436-4ef7-a921-e518b686e57e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eccee94872840afacff642b7d05cb72', 'request_id': 'req-f3f6d176-9436-4ef7-a921-e518b686e57e'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:26.192362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f3f6d176-9436-4ef7-a921-e518b686e57e admin admin] The token cache is being invalidate because project 5eccee94872840afacff642b7d05cb72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:26.192649 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f3f6d176-9436-4ef7-a921-e518b686e57e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:26.197146 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f3f6d176-9436-4ef7-a921-e518b686e57e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:26.202413 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 954/1913] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:26 2022] DELETE /identity/v3/projects/5eccee94872840afacff642b7d05cb72 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:26.203936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ae092-eb39-4ec8-9b78-b6aa7f5ee3e0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:26.204225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ae092-eb39-4ec8-9b78-b6aa7f5ee3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:26.204499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ae092-eb39-4ec8-9b78-b6aa7f5ee3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:26.228378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-8d2ae092-eb39-4ec8-9b78-b6aa7f5ee3e0 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:26.230301 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8d2ae092-eb39-4ec8-9b78-b6aa7f5ee3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:26.290829 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 960/1914] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:26.298832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677ebe7-bb86-4407-bba7-e426e7bde6b4 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:26.299114 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677ebe7-bb86-4407-bba7-e426e7bde6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:26.299387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677ebe7-bb86-4407-bba7-e426e7bde6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:26.319798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-6677ebe7-bb86-4407-bba7-e426e7bde6b4 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:26.321635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6677ebe7-bb86-4407-bba7-e426e7bde6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:26.329431 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 955/1915] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:26.333638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ae092-eb39-4ec8-9b78-b6aa7f5ee3e0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:26.343439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ae092-eb39-4ec8-9b78-b6aa7f5ee3e0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:26.348094 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ce4fe-deb2-4348-964c-e8428a10014f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:26.348693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ce4fe-deb2-4348-964c-e8428a10014f admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:26.348972 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ce4fe-deb2-4348-964c-e8428a10014f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:26.349248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ce4fe-deb2-4348-964c-e8428a10014f admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:26.349889 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ce4fe-deb2-4348-964c-e8428a10014f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:07:26.350226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ce4fe-deb2-4348-964c-e8428a10014f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:26.351825 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ce4fe-deb2-4348-964c-e8428a10014f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:26.363630 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 961/1916] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:07:26.368475 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677ebe7-bb86-4407-bba7-e426e7bde6b4 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:26.378497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677ebe7-bb86-4407-bba7-e426e7bde6b4 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:26.383790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa536eb6-1698-4b76-ab0b-a5fa6a70ca29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:26.384484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa536eb6-1698-4b76-ab0b-a5fa6a70ca29 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:26.384824 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa536eb6-1698-4b76-ab0b-a5fa6a70ca29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:26.385154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa536eb6-1698-4b76-ab0b-a5fa6a70ca29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:26.385883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa536eb6-1698-4b76-ab0b-a5fa6a70ca29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:26.387910 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa536eb6-1698-4b76-ab0b-a5fa6a70ca29 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:26.409475 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:26.421003 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 956/1917] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:07:26 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:26.422267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ce4fe-deb2-4348-964c-e8428a10014f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:26.434526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ce4fe-deb2-4348-964c-e8428a10014f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:26.452517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:26.504994 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7624f-00f5-4761-ad0f-2ba2086b5c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:26.505748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7624f-00f5-4761-ad0f-2ba2086b5c2e admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:26.506026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7624f-00f5-4761-ad0f-2ba2086b5c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:26.506332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7624f-00f5-4761-ad0f-2ba2086b5c2e admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:26.506968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7624f-00f5-4761-ad0f-2ba2086b5c2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:26.509463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7624f-00f5-4761-ad0f-2ba2086b5c2e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:26.535775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 962/1918] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:07:26 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:26.537001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa536eb6-1698-4b76-ab0b-a5fa6a70ca29 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:26.548509 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa536eb6-1698-4b76-ab0b-a5fa6a70ca29 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:26.553170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064e54a-9d29-49d6-9160-13a4baa67538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:26.553775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064e54a-9d29-49d6-9160-13a4baa67538 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:26.554050 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064e54a-9d29-49d6-9160-13a4baa67538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:26.554344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064e54a-9d29-49d6-9160-13a4baa67538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:26.555646 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064e54a-9d29-49d6-9160-13a4baa67538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:26.557238 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064e54a-9d29-49d6-9160-13a4baa67538 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:26.567776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 957/1919] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:07:26.572467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7624f-00f5-4761-ad0f-2ba2086b5c2e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:26.583086 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7624f-00f5-4761-ad0f-2ba2086b5c2e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:26.587993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff498c9c-11da-4a04-8d28-fcf221c794c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:26.588601 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff498c9c-11da-4a04-8d28-fcf221c794c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:26.588877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff498c9c-11da-4a04-8d28-fcf221c794c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:26.589146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff498c9c-11da-4a04-8d28-fcf221c794c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:26.589784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff498c9c-11da-4a04-8d28-fcf221c794c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:26.591679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff498c9c-11da-4a04-8d28-fcf221c794c0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:26.602194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 963/1920] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:26.606608 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064e54a-9d29-49d6-9160-13a4baa67538 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:26.617635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064e54a-9d29-49d6-9160-13a4baa67538 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:26.622765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190366af-e4b2-4193-b7a8-d686a43ddd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:26.623431 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190366af-e4b2-4193-b7a8-d686a43ddd18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:26.623761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190366af-e4b2-4193-b7a8-d686a43ddd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:26.623995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190366af-e4b2-4193-b7a8-d686a43ddd18 admin admin] PATH_INFO: `/v3/projects/daf03b965c864fcaae2dbd6bcc5250ce/users/de2545b9850648afaab1077172893c3d/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:26.640881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190366af-e4b2-4193-b7a8-d686a43ddd18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf03b965c864fcaae2dbd6bcc5250ce, user_id=de2545b9850648afaab1077172893c3d, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:26.644907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190366af-e4b2-4193-b7a8-d686a43ddd18 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:26.659326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 958/1921] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:26 2022] PUT /identity/v3/projects/daf03b965c864fcaae2dbd6bcc5250ce/users/de2545b9850648afaab1077172893c3d/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:26.670698 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c6f55-e9dd-4ecf-8d68-9c8fe5eb07fe None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:26.671275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c6f55-e9dd-4ecf-8d68-9c8fe5eb07fe None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:26.671275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c6f55-e9dd-4ecf-8d68-9c8fe5eb07fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:26.701332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-794c6f55-e9dd-4ecf-8d68-9c8fe5eb07fe None None] MFA Rules not processed for user `de2545b9850648afaab1077172893c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:26.702673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-794c6f55-e9dd-4ecf-8d68-9c8fe5eb07fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:26.775686 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 964/1922] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:26.786880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190366af-e4b2-4193-b7a8-d686a43ddd18 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:26.788593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-190366af-e4b2-4193-b7a8-d686a43ddd18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:26.846936 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190366af-e4b2-4193-b7a8-d686a43ddd18 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:26.850623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba1b8b-45a5-4ad8-843c-d13f9fc884ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:26.851226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba1b8b-45a5-4ad8-843c-d13f9fc884ec service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:26.852002 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba1b8b-45a5-4ad8-843c-d13f9fc884ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:26.852325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba1b8b-45a5-4ad8-843c-d13f9fc884ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:26.860494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba1b8b-45a5-4ad8-843c-d13f9fc884ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:26.861912 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba1b8b-45a5-4ad8-843c-d13f9fc884ec service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:26.871621 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 959/1923] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:26.893492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c6f55-e9dd-4ecf-8d68-9c8fe5eb07fe None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:26.957095 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c6f55-e9dd-4ecf-8d68-9c8fe5eb07fe None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:26.961875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536baaf-5c8d-49eb-83b6-406cc9c78f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:26.962580 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536baaf-5c8d-49eb-83b6-406cc9c78f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:26.962861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536baaf-5c8d-49eb-83b6-406cc9c78f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:26.963136 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536baaf-5c8d-49eb-83b6-406cc9c78f5c admin admin] PATH_INFO: `/v3/users/de2545b9850648afaab1077172893c3d` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:26.964409 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536baaf-5c8d-49eb-83b6-406cc9c78f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=de2545b9850648afaab1077172893c3d)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:26.966422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536baaf-5c8d-49eb-83b6-406cc9c78f5c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:26.999393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:27.019867 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a536baaf-5c8d-49eb-83b6-406cc9c78f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de2545b9850648afaab1077172893c3d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:27.020562 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a536baaf-5c8d-49eb-83b6-406cc9c78f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de2545b9850648afaab1077172893c3d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:27.028405 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a536baaf-5c8d-49eb-83b6-406cc9c78f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de2545b9850648afaab1077172893c3d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:27.037976 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a536baaf-5c8d-49eb-83b6-406cc9c78f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de2545b9850648afaab1077172893c3d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:27.052248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a536baaf-5c8d-49eb-83b6-406cc9c78f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de2545b9850648afaab1077172893c3d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:27.058781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 965/1924] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:26 2022] DELETE /identity/v3/users/de2545b9850648afaab1077172893c3d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:27.063432 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dde664-ff80-4996-945c-265bf3f83c7d None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:27.063833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dde664-ff80-4996-945c-265bf3f83c7d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:27.064180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dde664-ff80-4996-945c-265bf3f83c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:27.084493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-09dde664-ff80-4996-945c-265bf3f83c7d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:27.086920 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-09dde664-ff80-4996-945c-265bf3f83c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:27.147646 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 960/1925] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:27.157996 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536baaf-5c8d-49eb-83b6-406cc9c78f5c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:27.159421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a536baaf-5c8d-49eb-83b6-406cc9c78f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:27.234506 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536baaf-5c8d-49eb-83b6-406cc9c78f5c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:27.234506 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1421a-016b-44d0-9f55-5056bc2a5e21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:27.234506 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1421a-016b-44d0-9f55-5056bc2a5e21 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:27.234506 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1421a-016b-44d0-9f55-5056bc2a5e21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:27.235313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1421a-016b-44d0-9f55-5056bc2a5e21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:27.246109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1421a-016b-44d0-9f55-5056bc2a5e21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:27.247974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1421a-016b-44d0-9f55-5056bc2a5e21 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:27.270320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 966/1926] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:27.558687 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dde664-ff80-4996-945c-265bf3f83c7d None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:27.560134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-09dde664-ff80-4996-945c-265bf3f83c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:27.577348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dde664-ff80-4996-945c-265bf3f83c7d None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:27.583584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c54ee2-e2d1-4c24-9372-c9b43e021bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:27.584265 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c54ee2-e2d1-4c24-9372-c9b43e021bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:27.584548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c54ee2-e2d1-4c24-9372-c9b43e021bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:27.584825 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c54ee2-e2d1-4c24-9372-c9b43e021bfb admin admin] PATH_INFO: `/v3/projects/daf03b965c864fcaae2dbd6bcc5250ce` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:27.586322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c54ee2-e2d1-4c24-9372-c9b43e021bfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=daf03b965c864fcaae2dbd6bcc5250ce)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:27.595403 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c54ee2-e2d1-4c24-9372-c9b43e021bfb admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:27.655749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-39c54ee2-e2d1-4c24-9372-c9b43e021bfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daf03b965c864fcaae2dbd6bcc5250ce', 'request_id': 'req-39c54ee2-e2d1-4c24-9372-c9b43e021bfb'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:27.664433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-39c54ee2-e2d1-4c24-9372-c9b43e021bfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daf03b965c864fcaae2dbd6bcc5250ce', 'request_id': 'req-39c54ee2-e2d1-4c24-9372-c9b43e021bfb'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:27.669491 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:27.671560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-39c54ee2-e2d1-4c24-9372-c9b43e021bfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daf03b965c864fcaae2dbd6bcc5250ce', 'request_id': 'req-39c54ee2-e2d1-4c24-9372-c9b43e021bfb'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:27.686721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-39c54ee2-e2d1-4c24-9372-c9b43e021bfb admin admin] The token cache is being invalidate because project daf03b965c864fcaae2dbd6bcc5250ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:27.687117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-39c54ee2-e2d1-4c24-9372-c9b43e021bfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:27.687843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-39c54ee2-e2d1-4c24-9372-c9b43e021bfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:27.689459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 961/1927] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:27 2022] DELETE /identity/v3/projects/daf03b965c864fcaae2dbd6bcc5250ce => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:27.695073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4a9af-9e1b-4e5c-aa05-fc32d5c14293 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:27.695604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4a9af-9e1b-4e5c-aa05-fc32d5c14293 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:27.695604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4a9af-9e1b-4e5c-aa05-fc32d5c14293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:27.719208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-23e4a9af-9e1b-4e5c-aa05-fc32d5c14293 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:27.720425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-23e4a9af-9e1b-4e5c-aa05-fc32d5c14293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:27.781488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 967/1928] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:27.790346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77becce7-0f5e-4558-8db0-606770bf1d43 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:27.790634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77becce7-0f5e-4558-8db0-606770bf1d43 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:27.790912 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77becce7-0f5e-4558-8db0-606770bf1d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:27.813120 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-77becce7-0f5e-4558-8db0-606770bf1d43 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:27.814396 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-77becce7-0f5e-4558-8db0-606770bf1d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:27.823511 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 962/1929] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:27.828275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4a9af-9e1b-4e5c-aa05-fc32d5c14293 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:27.839333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4a9af-9e1b-4e5c-aa05-fc32d5c14293 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:27.847054 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d08025-96a1-47af-8d61-d2264bb9ec3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:27.847692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d08025-96a1-47af-8d61-d2264bb9ec3a admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:27.847968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d08025-96a1-47af-8d61-d2264bb9ec3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:27.848236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d08025-96a1-47af-8d61-d2264bb9ec3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:27.848893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d08025-96a1-47af-8d61-d2264bb9ec3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:07:27.849224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d08025-96a1-47af-8d61-d2264bb9ec3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:27.850869 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d08025-96a1-47af-8d61-d2264bb9ec3a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:27.867709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 968/1930] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:07:27.869158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77becce7-0f5e-4558-8db0-606770bf1d43 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:27.879612 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77becce7-0f5e-4558-8db0-606770bf1d43 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:27.885500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f890ce-9dc6-4b48-a9b0-3c832928aab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:27.886172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f890ce-9dc6-4b48-a9b0-3c832928aab7 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:27.886458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f890ce-9dc6-4b48-a9b0-3c832928aab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:27.886733 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f890ce-9dc6-4b48-a9b0-3c832928aab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:27.887430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f890ce-9dc6-4b48-a9b0-3c832928aab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:27.889533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f890ce-9dc6-4b48-a9b0-3c832928aab7 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:27.919732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 963/1931] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:07:27 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:27.924275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d08025-96a1-47af-8d61-d2264bb9ec3a admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:27.935935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d08025-96a1-47af-8d61-d2264bb9ec3a admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:27.980461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:28.013638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac90e8-b2b4-4d7d-9272-35fd17a5cc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:28.014811 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac90e8-b2b4-4d7d-9272-35fd17a5cc05 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:28.015232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac90e8-b2b4-4d7d-9272-35fd17a5cc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:28.015666 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac90e8-b2b4-4d7d-9272-35fd17a5cc05 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:28.016809 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac90e8-b2b4-4d7d-9272-35fd17a5cc05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:28.019102 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac90e8-b2b4-4d7d-9272-35fd17a5cc05 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:28.042768 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 969/1932] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:07:27 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:28.046887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f890ce-9dc6-4b48-a9b0-3c832928aab7 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:28.058467 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f890ce-9dc6-4b48-a9b0-3c832928aab7 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:28.063445 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853930a-3133-4ba8-9a22-956884c1f8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:28.064045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853930a-3133-4ba8-9a22-956884c1f8ec admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:28.064326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853930a-3133-4ba8-9a22-956884c1f8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:28.064598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853930a-3133-4ba8-9a22-956884c1f8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:28.065247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853930a-3133-4ba8-9a22-956884c1f8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:28.067341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853930a-3133-4ba8-9a22-956884c1f8ec admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:28.078880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 964/1933] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:28.083032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac90e8-b2b4-4d7d-9272-35fd17a5cc05 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:28.093733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac90e8-b2b4-4d7d-9272-35fd17a5cc05 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:28.098951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464e719-f1f2-4598-bb71-ef9d8e309b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:28.099583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464e719-f1f2-4598-bb71-ef9d8e309b9c admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:28.099867 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464e719-f1f2-4598-bb71-ef9d8e309b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:28.100139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464e719-f1f2-4598-bb71-ef9d8e309b9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:28.100806 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464e719-f1f2-4598-bb71-ef9d8e309b9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:28.102527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464e719-f1f2-4598-bb71-ef9d8e309b9c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:28.114199 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 970/1934] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:28.119342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853930a-3133-4ba8-9a22-956884c1f8ec admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:28.130440 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853930a-3133-4ba8-9a22-956884c1f8ec admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:28.135972 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d15357-d7a8-42f5-8859-fa3607f67b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:28.136657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d15357-d7a8-42f5-8859-fa3607f67b0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:28.136940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d15357-d7a8-42f5-8859-fa3607f67b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:28.137215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d15357-d7a8-42f5-8859-fa3607f67b0c admin admin] PATH_INFO: `/v3/projects/49da20621aec468398e6e1ad654e62bb/users/196df48558094813af62bc26ac2c4a91/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:28.156389 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d15357-d7a8-42f5-8859-fa3607f67b0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=49da20621aec468398e6e1ad654e62bb, user_id=196df48558094813af62bc26ac2c4a91, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:28.159471 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d15357-d7a8-42f5-8859-fa3607f67b0c admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:28.175040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 965/1935] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:28 2022] PUT /identity/v3/projects/49da20621aec468398e6e1ad654e62bb/users/196df48558094813af62bc26ac2c4a91/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:28.187036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273fb48-a599-4f7e-8e79-1e60ba929a8a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:28.187433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273fb48-a599-4f7e-8e79-1e60ba929a8a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:28.187719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273fb48-a599-4f7e-8e79-1e60ba929a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:28.218435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a273fb48-a599-4f7e-8e79-1e60ba929a8a None None] MFA Rules not processed for user `196df48558094813af62bc26ac2c4a91`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:28.219660 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a273fb48-a599-4f7e-8e79-1e60ba929a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:28.303804 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 971/1936] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:28.308850 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d15357-d7a8-42f5-8859-fa3607f67b0c admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:28.389665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d15357-d7a8-42f5-8859-fa3607f67b0c admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:28.395443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61568fb1-96e3-45fc-bc73-8d54cb236b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:28.396183 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61568fb1-96e3-45fc-bc73-8d54cb236b94 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:28.396467 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61568fb1-96e3-45fc-bc73-8d54cb236b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:28.396740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61568fb1-96e3-45fc-bc73-8d54cb236b94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:28.397394 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61568fb1-96e3-45fc-bc73-8d54cb236b94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:28.399849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61568fb1-96e3-45fc-bc73-8d54cb236b94 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:28.437355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 966/1937] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:07:28 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:28.446233 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273fb48-a599-4f7e-8e79-1e60ba929a8a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:28.457727 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273fb48-a599-4f7e-8e79-1e60ba929a8a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:28.549450 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aea591-39bb-462c-9545-063d4482c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:28.549975 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aea591-39bb-462c-9545-063d4482c442 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:28.550278 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aea591-39bb-462c-9545-063d4482c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:28.550550 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aea591-39bb-462c-9545-063d4482c442 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:28.551206 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aea591-39bb-462c-9545-063d4482c442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:28.557032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aea591-39bb-462c-9545-063d4482c442 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:28.589981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 972/1938] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:07:28 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:28.595820 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61568fb1-96e3-45fc-bc73-8d54cb236b94 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:28.610421 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61568fb1-96e3-45fc-bc73-8d54cb236b94 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:28.616520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccfc04-5ffd-48cc-9919-51f33d5ae57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:28.617668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccfc04-5ffd-48cc-9919-51f33d5ae57b admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:28.617947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccfc04-5ffd-48cc-9919-51f33d5ae57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:28.618463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccfc04-5ffd-48cc-9919-51f33d5ae57b admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:28.619129 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fccfc04-5ffd-48cc-9919-51f33d5ae57b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:28.620996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fccfc04-5ffd-48cc-9919-51f33d5ae57b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:28.635162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 967/1939] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:28.639835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aea591-39bb-462c-9545-063d4482c442 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:28.655519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aea591-39bb-462c-9545-063d4482c442 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:28.663884 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cdb72-0490-447a-8f44-86552af5480d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:28.664831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cdb72-0490-447a-8f44-86552af5480d admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:28.665220 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cdb72-0490-447a-8f44-86552af5480d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:28.665766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cdb72-0490-447a-8f44-86552af5480d admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:28.666631 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cdb72-0490-447a-8f44-86552af5480d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:28.668538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cdb72-0490-447a-8f44-86552af5480d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:28.697963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 973/1940] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:28.703472 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccfc04-5ffd-48cc-9919-51f33d5ae57b admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:28.720782 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccfc04-5ffd-48cc-9919-51f33d5ae57b admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:28.727786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd4ed4-2bab-487b-aa72-204131681ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:28.728586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd4ed4-2bab-487b-aa72-204131681ae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:28.728926 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd4ed4-2bab-487b-aa72-204131681ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:28.729267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd4ed4-2bab-487b-aa72-204131681ae3 admin admin] PATH_INFO: `/v3/projects/a64dc82743ff4b259a52c3985a5fb3e5/users/6e79f5549c1246e0becb76f7bbc24f91/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:28.766110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd4ed4-2bab-487b-aa72-204131681ae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a64dc82743ff4b259a52c3985a5fb3e5, user_id=6e79f5549c1246e0becb76f7bbc24f91, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:28.774971 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd4ed4-2bab-487b-aa72-204131681ae3 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:28.807287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 968/1941] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:28 2022] PUT /identity/v3/projects/a64dc82743ff4b259a52c3985a5fb3e5/users/6e79f5549c1246e0becb76f7bbc24f91/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:28.819993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5072a-9dbb-4acb-8987-5a269ba65a2f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:28.820414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5072a-9dbb-4acb-8987-5a269ba65a2f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:28.820681 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5072a-9dbb-4acb-8987-5a269ba65a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:28.860992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d1b5072a-9dbb-4acb-8987-5a269ba65a2f None None] MFA Rules not processed for user `6e79f5549c1246e0becb76f7bbc24f91`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:28.862386 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d1b5072a-9dbb-4acb-8987-5a269ba65a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:28.879872 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:28.930576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd4ed4-2bab-487b-aa72-204131681ae3 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:28.933425 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-8fdd4ed4-2bab-487b-aa72-204131681ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:28.999911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 974/1942] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:29.001962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5072a-9dbb-4acb-8987-5a269ba65a2f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:29.058650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:29.128857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd4ed4-2bab-487b-aa72-204131681ae3 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:29.145331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47efc6-9f11-4682-81ab-a592e3037bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:29.146277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47efc6-9f11-4682-81ab-a592e3037bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:29.146432 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47efc6-9f11-4682-81ab-a592e3037bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:29.146710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47efc6-9f11-4682-81ab-a592e3037bc4 admin admin] PATH_INFO: `/v3/users/cead08c2f8ed4b918419804d705f8fe0` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:29.151865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47efc6-9f11-4682-81ab-a592e3037bc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cead08c2f8ed4b918419804d705f8fe0)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:29.154071 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47efc6-9f11-4682-81ab-a592e3037bc4 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:29.194622 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5072a-9dbb-4acb-8987-5a269ba65a2f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:29.207545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080203f0-ae0e-4de5-99e4-ed0c803736c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:29.208180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080203f0-ae0e-4de5-99e4-ed0c803736c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:29.208457 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080203f0-ae0e-4de5-99e4-ed0c803736c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:29.208862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080203f0-ae0e-4de5-99e4-ed0c803736c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:29.209509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080203f0-ae0e-4de5-99e4-ed0c803736c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:29.212410 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080203f0-ae0e-4de5-99e4-ed0c803736c0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:29.247019 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bd47efc6-9f11-4682-81ab-a592e3037bc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cead08c2f8ed4b918419804d705f8fe0'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:29.258015 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bd47efc6-9f11-4682-81ab-a592e3037bc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cead08c2f8ed4b918419804d705f8fe0'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:29.262520 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 975/1943] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:07:29 2022] POST /identity/v3/projects => generated 383 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:29.266595 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080203f0-ae0e-4de5-99e4-ed0c803736c0 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:29.285380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bd47efc6-9f11-4682-81ab-a592e3037bc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cead08c2f8ed4b918419804d705f8fe0'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:29.295039 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bd47efc6-9f11-4682-81ab-a592e3037bc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cead08c2f8ed4b918419804d705f8fe0'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:29.296151 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bd47efc6-9f11-4682-81ab-a592e3037bc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cead08c2f8ed4b918419804d705f8fe0'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:29.304975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 969/1944] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:28 2022] DELETE /identity/v3/users/cead08c2f8ed4b918419804d705f8fe0 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:29.311913 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74025d80-58e5-4866-a1d5-fbe4c0f237f0 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:29.312222 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74025d80-58e5-4866-a1d5-fbe4c0f237f0 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:29.312477 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74025d80-58e5-4866-a1d5-fbe4c0f237f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:29.337821 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-74025d80-58e5-4866-a1d5-fbe4c0f237f0 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:29.339163 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-74025d80-58e5-4866-a1d5-fbe4c0f237f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:29.424332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080203f0-ae0e-4de5-99e4-ed0c803736c0 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:29.518848 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 970/1945] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:29.526603 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:29.553385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507479a-f2af-465a-827f-27472b225ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:29.556991 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74025d80-58e5-4866-a1d5-fbe4c0f237f0 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:29.559695 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-74025d80-58e5-4866-a1d5-fbe4c0f237f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:29.568199 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507479a-f2af-465a-827f-27472b225ace admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:29.570691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507479a-f2af-465a-827f-27472b225ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:29.573056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507479a-f2af-465a-827f-27472b225ace admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:29.576124 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507479a-f2af-465a-827f-27472b225ace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:29.581073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507479a-f2af-465a-827f-27472b225ace admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:29.643191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 976/1946] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:07:29 2022] POST /identity/v3/users => generated 298 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:29.656178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507479a-f2af-465a-827f-27472b225ace admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:29.659473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2507479a-f2af-465a-827f-27472b225ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:29.681921 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507479a-f2af-465a-827f-27472b225ace admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:29.694832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d9e43-285e-4252-8c02-5ae75d225c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:29.696270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d9e43-285e-4252-8c02-5ae75d225c37 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:29.696946 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d9e43-285e-4252-8c02-5ae75d225c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:29.697433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d9e43-285e-4252-8c02-5ae75d225c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:29.699779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d9e43-285e-4252-8c02-5ae75d225c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:29.703708 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74025d80-58e5-4866-a1d5-fbe4c0f237f0 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:29.704822 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d9e43-285e-4252-8c02-5ae75d225c37 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:29.715701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05622a6-138c-4be3-8672-9a3870e82d0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:29.717357 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05622a6-138c-4be3-8672-9a3870e82d0e service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:29.717357 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05622a6-138c-4be3-8672-9a3870e82d0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:29.717357 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05622a6-138c-4be3-8672-9a3870e82d0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:29.725247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 977/1947] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:29.730439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d9e43-285e-4252-8c02-5ae75d225c37 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:29.740732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05622a6-138c-4be3-8672-9a3870e82d0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:29.742311 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05622a6-138c-4be3-8672-9a3870e82d0e service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:29.751914 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d9e43-285e-4252-8c02-5ae75d225c37 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:29.753756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:29.764688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab746d7e-2c7f-4a3e-bafd-ab1412bad42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:29.766207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab746d7e-2c7f-4a3e-bafd-ab1412bad42f admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:29.766786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 971/1948] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:29.767137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab746d7e-2c7f-4a3e-bafd-ab1412bad42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:29.767640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab746d7e-2c7f-4a3e-bafd-ab1412bad42f admin admin] PATH_INFO: `/v3/projects/8589ada6a70b4430b07accaf57d9f2d9/users/351355bb03cb4268b834300db5bb9b1b/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:29.800210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab746d7e-2c7f-4a3e-bafd-ab1412bad42f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8589ada6a70b4430b07accaf57d9f2d9, user_id=351355bb03cb4268b834300db5bb9b1b, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:29.807250 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab746d7e-2c7f-4a3e-bafd-ab1412bad42f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:29.836368 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 978/1949] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:29 2022] PUT /identity/v3/projects/8589ada6a70b4430b07accaf57d9f2d9/users/351355bb03cb4268b834300db5bb9b1b/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:29.840615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05622a6-138c-4be3-8672-9a3870e82d0e service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:29.851815 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05622a6-138c-4be3-8672-9a3870e82d0e service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:29.927354 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84799f-66bd-4fef-8826-dd96dc43f1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:29.928248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84799f-66bd-4fef-8826-dd96dc43f1fe admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:29.928718 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84799f-66bd-4fef-8826-dd96dc43f1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:29.929151 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84799f-66bd-4fef-8826-dd96dc43f1fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:29.930447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84799f-66bd-4fef-8826-dd96dc43f1fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:29.932219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84799f-66bd-4fef-8826-dd96dc43f1fe admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:29.951889 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 972/1950] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:29.953679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab746d7e-2c7f-4a3e-bafd-ab1412bad42f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:29.965823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab746d7e-2c7f-4a3e-bafd-ab1412bad42f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:29.972590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2953142-c148-4539-ad35-81fa2ff3e282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:29.973237 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2953142-c148-4539-ad35-81fa2ff3e282 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:29.973531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2953142-c148-4539-ad35-81fa2ff3e282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:29.973816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2953142-c148-4539-ad35-81fa2ff3e282 admin admin] PATH_INFO: `/v3/projects/8589ada6a70b4430b07accaf57d9f2d9/users/351355bb03cb4268b834300db5bb9b1b/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:29.976798 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2953142-c148-4539-ad35-81fa2ff3e282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8589ada6a70b4430b07accaf57d9f2d9, user_id=351355bb03cb4268b834300db5bb9b1b, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:29.979194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2953142-c148-4539-ad35-81fa2ff3e282 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:29.993165 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 979/1951] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:29 2022] PUT /identity/v3/projects/8589ada6a70b4430b07accaf57d9f2d9/users/351355bb03cb4268b834300db5bb9b1b/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:30.070685 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84b11d-e8f7-4196-b202-b8a3a4b9c979 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:30.070685 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84b11d-e8f7-4196-b202-b8a3a4b9c979 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:30.070685 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84b11d-e8f7-4196-b202-b8a3a4b9c979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:30.105905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-fe84b11d-e8f7-4196-b202-b8a3a4b9c979 None None] MFA Rules not processed for user `351355bb03cb4268b834300db5bb9b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:30.107214 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-fe84b11d-e8f7-4196-b202-b8a3a4b9c979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:30.215377 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 973/1952] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:30 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:30.237075 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2953142-c148-4539-ad35-81fa2ff3e282 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:30.239116 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a2953142-c148-4539-ad35-81fa2ff3e282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:30.304059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2953142-c148-4539-ad35-81fa2ff3e282 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:30.307818 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d2d70-8b10-4a11-b270-c6a3b27f591d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:30.308453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d2d70-8b10-4a11-b270-c6a3b27f591d service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:30.308731 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d2d70-8b10-4a11-b270-c6a3b27f591d service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:30.309008 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d2d70-8b10-4a11-b270-c6a3b27f591d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:30.311040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4d9d2d70-8b10-4a11-b270-c6a3b27f591d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:30.383321 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d2d70-8b10-4a11-b270-c6a3b27f591d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:30.384569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d2d70-8b10-4a11-b270-c6a3b27f591d service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:30.391596 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84b11d-e8f7-4196-b202-b8a3a4b9c979 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:30.393431 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-fe84b11d-e8f7-4196-b202-b8a3a4b9c979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:30.394491 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 980/1953] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:30.507202 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84b11d-e8f7-4196-b202-b8a3a4b9c979 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:30.513000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a554bb2-931a-4f60-bf28-179576c68389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:30.513654 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a554bb2-931a-4f60-bf28-179576c68389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:30.513935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a554bb2-931a-4f60-bf28-179576c68389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:30.514237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a554bb2-931a-4f60-bf28-179576c68389 admin admin] PATH_INFO: `/v3/projects/f5536985a156470cbb865649fd1abfbd` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:30.515388 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a554bb2-931a-4f60-bf28-179576c68389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5536985a156470cbb865649fd1abfbd)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:30.517465 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a554bb2-931a-4f60-bf28-179576c68389 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:30.545554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:30.599963 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7a554bb2-931a-4f60-bf28-179576c68389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5536985a156470cbb865649fd1abfbd', 'request_id': 'req-7a554bb2-931a-4f60-bf28-179576c68389'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:30.612873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7a554bb2-931a-4f60-bf28-179576c68389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5536985a156470cbb865649fd1abfbd', 'request_id': 'req-7a554bb2-931a-4f60-bf28-179576c68389'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:30.621901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7a554bb2-931a-4f60-bf28-179576c68389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5536985a156470cbb865649fd1abfbd', 'request_id': 'req-7a554bb2-931a-4f60-bf28-179576c68389'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:30.653269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7a554bb2-931a-4f60-bf28-179576c68389 admin admin] The token cache is being invalidate because project f5536985a156470cbb865649fd1abfbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:30.653751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7a554bb2-931a-4f60-bf28-179576c68389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:30.670968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7a554bb2-931a-4f60-bf28-179576c68389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:30.674718 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 974/1954] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:30 2022] DELETE /identity/v3/projects/f5536985a156470cbb865649fd1abfbd => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:07:30.680077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d2d70-8b10-4a11-b270-c6a3b27f591d service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:30.694596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d2d70-8b10-4a11-b270-c6a3b27f591d service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:30.761221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:30.785182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2789211-6364-4164-a730-312a0d6f0290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:30.786042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2789211-6364-4164-a730-312a0d6f0290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:30.786449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2789211-6364-4164-a730-312a0d6f0290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:30.786847 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2789211-6364-4164-a730-312a0d6f0290 admin admin] PATH_INFO: `/v3/users/c4f5885898f6420d83d8558af7343114` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:30.788171 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2789211-6364-4164-a730-312a0d6f0290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4f5885898f6420d83d8558af7343114)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:30.790922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2789211-6364-4164-a730-312a0d6f0290 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:30.865624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b2789211-6364-4164-a730-312a0d6f0290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4f5885898f6420d83d8558af7343114'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:30.866571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b2789211-6364-4164-a730-312a0d6f0290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4f5885898f6420d83d8558af7343114'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:30.896538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b2789211-6364-4164-a730-312a0d6f0290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4f5885898f6420d83d8558af7343114'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:30.909938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b2789211-6364-4164-a730-312a0d6f0290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4f5885898f6420d83d8558af7343114'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:30.932850 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b2789211-6364-4164-a730-312a0d6f0290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4f5885898f6420d83d8558af7343114'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:30.946142 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 981/1955] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:30 2022] DELETE /identity/v3/users/c4f5885898f6420d83d8558af7343114 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:07:31.407108 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a554bb2-931a-4f60-bf28-179576c68389 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:31.408663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7a554bb2-931a-4f60-bf28-179576c68389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:31.427665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a554bb2-931a-4f60-bf28-179576c68389 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:31.503291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973b174-5707-4105-9f8a-b9fc88e1ea78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:31.503929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973b174-5707-4105-9f8a-b9fc88e1ea78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:31.504686 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973b174-5707-4105-9f8a-b9fc88e1ea78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:31.504686 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973b174-5707-4105-9f8a-b9fc88e1ea78 admin admin] PATH_INFO: `/v3/projects/59fff47f3e034672b3e0e523558d9933` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:31.506207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973b174-5707-4105-9f8a-b9fc88e1ea78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59fff47f3e034672b3e0e523558d9933)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:31.508620 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973b174-5707-4105-9f8a-b9fc88e1ea78 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:31.555609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0973b174-5707-4105-9f8a-b9fc88e1ea78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59fff47f3e034672b3e0e523558d9933', 'request_id': 'req-0973b174-5707-4105-9f8a-b9fc88e1ea78'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:31.562952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0973b174-5707-4105-9f8a-b9fc88e1ea78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59fff47f3e034672b3e0e523558d9933', 'request_id': 'req-0973b174-5707-4105-9f8a-b9fc88e1ea78'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:31.568646 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0973b174-5707-4105-9f8a-b9fc88e1ea78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59fff47f3e034672b3e0e523558d9933', 'request_id': 'req-0973b174-5707-4105-9f8a-b9fc88e1ea78'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:31.581610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0973b174-5707-4105-9f8a-b9fc88e1ea78 admin admin] The token cache is being invalidate because project 59fff47f3e034672b3e0e523558d9933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:31.581921 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0973b174-5707-4105-9f8a-b9fc88e1ea78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:31.582859 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0973b174-5707-4105-9f8a-b9fc88e1ea78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:31.584735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 975/1956] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:31 2022] DELETE /identity/v3/projects/59fff47f3e034672b3e0e523558d9933 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:31.589919 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0262361-2782-4a22-9aaa-acb8b4e76b44 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:31.590639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0262361-2782-4a22-9aaa-acb8b4e76b44 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:31.590931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0262361-2782-4a22-9aaa-acb8b4e76b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:31.611507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d0262361-2782-4a22-9aaa-acb8b4e76b44 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:31.612712 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d0262361-2782-4a22-9aaa-acb8b4e76b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:31.673333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 982/1957] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:31.685691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b308c0-80f7-4c35-9feb-464ccee7322e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:31.686049 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b308c0-80f7-4c35-9feb-464ccee7322e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:31.686287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b308c0-80f7-4c35-9feb-464ccee7322e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:31.706925 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-e7b308c0-80f7-4c35-9feb-464ccee7322e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:31.708173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e7b308c0-80f7-4c35-9feb-464ccee7322e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:31.716249 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 976/1958] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:31.720633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0262361-2782-4a22-9aaa-acb8b4e76b44 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:31.731408 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0262361-2782-4a22-9aaa-acb8b4e76b44 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:31.738082 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6c890-028e-451d-9a6c-65c9a0896d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:31.739210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6c890-028e-451d-9a6c-65c9a0896d0e admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:31.739804 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6c890-028e-451d-9a6c-65c9a0896d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:31.740255 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6c890-028e-451d-9a6c-65c9a0896d0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:31.741535 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6c890-028e-451d-9a6c-65c9a0896d0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:07:31.742548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6c890-028e-451d-9a6c-65c9a0896d0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:31.744465 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6c890-028e-451d-9a6c-65c9a0896d0e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:31.758049 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 983/1959] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:07:31.762522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b308c0-80f7-4c35-9feb-464ccee7322e None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:31.772407 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b308c0-80f7-4c35-9feb-464ccee7322e None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:31.777355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7ee01-b01f-47fd-a840-2a0fbaeb78ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:31.778216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7ee01-b01f-47fd-a840-2a0fbaeb78ff admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:31.778681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7ee01-b01f-47fd-a840-2a0fbaeb78ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:31.779103 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7ee01-b01f-47fd-a840-2a0fbaeb78ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:31.779977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7ee01-b01f-47fd-a840-2a0fbaeb78ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:31.782144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7ee01-b01f-47fd-a840-2a0fbaeb78ff admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:31.818951 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 977/1960] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:07:31 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:31.823182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6c890-028e-451d-9a6c-65c9a0896d0e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:31.835094 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6c890-028e-451d-9a6c-65c9a0896d0e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:31.896322 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:31.919413 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e0b63-63aa-4044-b6f0-c5f1814b1c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:31.920050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e0b63-63aa-4044-b6f0-c5f1814b1c94 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:31.920466 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e0b63-63aa-4044-b6f0-c5f1814b1c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:31.920744 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e0b63-63aa-4044-b6f0-c5f1814b1c94 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:31.921561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e0b63-63aa-4044-b6f0-c5f1814b1c94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:31.923616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e0b63-63aa-4044-b6f0-c5f1814b1c94 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:31.963760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 984/1961] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:07:31 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:31.967759 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7ee01-b01f-47fd-a840-2a0fbaeb78ff admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:31.991680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7ee01-b01f-47fd-a840-2a0fbaeb78ff admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:32.002561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059ebb4-2ca2-4b43-806c-4137a9f15e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:32.003763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059ebb4-2ca2-4b43-806c-4137a9f15e13 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:32.004429 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059ebb4-2ca2-4b43-806c-4137a9f15e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:32.004816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059ebb4-2ca2-4b43-806c-4137a9f15e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:32.005666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059ebb4-2ca2-4b43-806c-4137a9f15e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:32.008052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059ebb4-2ca2-4b43-806c-4137a9f15e13 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:32.014737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:32.025025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 978/1962] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:32.030423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e0b63-63aa-4044-b6f0-c5f1814b1c94 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:32.055080 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e0b63-63aa-4044-b6f0-c5f1814b1c94 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:32.076082 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2e6b0-7d96-450d-b2b1-0bba6e751432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:32.076713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2e6b0-7d96-450d-b2b1-0bba6e751432 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:32.076986 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2e6b0-7d96-450d-b2b1-0bba6e751432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:32.077255 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2e6b0-7d96-450d-b2b1-0bba6e751432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:32.077910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2e6b0-7d96-450d-b2b1-0bba6e751432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:32.087472 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2e6b0-7d96-450d-b2b1-0bba6e751432 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:32.130062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 985/1963] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:32.134896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059ebb4-2ca2-4b43-806c-4137a9f15e13 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:32.150750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059ebb4-2ca2-4b43-806c-4137a9f15e13 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:32.172480 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b5322-0218-4c9c-b161-5c8346809542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:32.174756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b5322-0218-4c9c-b161-5c8346809542 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:32.175523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b5322-0218-4c9c-b161-5c8346809542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:32.180438 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b5322-0218-4c9c-b161-5c8346809542 admin admin] PATH_INFO: `/v3/projects/e6bc2bbe33934b53ae1c6d15432fa2ea/users/bf714e0f43924fa5b1c8cc6f8c5295c8/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:32.182657 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2e6b0-7d96-450d-b2b1-0bba6e751432 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:32.185396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-dba2e6b0-7d96-450d-b2b1-0bba6e751432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:32.211266 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b5322-0218-4c9c-b161-5c8346809542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6bc2bbe33934b53ae1c6d15432fa2ea, user_id=bf714e0f43924fa5b1c8cc6f8c5295c8, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:32.216582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b5322-0218-4c9c-b161-5c8346809542 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:32.242443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 979/1964] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:32 2022] PUT /identity/v3/projects/e6bc2bbe33934b53ae1c6d15432fa2ea/users/bf714e0f43924fa5b1c8cc6f8c5295c8/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:32.243666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b5322-0218-4c9c-b161-5c8346809542 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:32.245640 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d74b5322-0218-4c9c-b161-5c8346809542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:32.301565 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2e6b0-7d96-450d-b2b1-0bba6e751432 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:32.350204 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b5322-0218-4c9c-b161-5c8346809542 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:32.355623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d18d7-d033-4ad2-a656-fc56ae22e860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:32.356613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d18d7-d033-4ad2-a656-fc56ae22e860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:32.356613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d18d7-d033-4ad2-a656-fc56ae22e860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:32.356870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d18d7-d033-4ad2-a656-fc56ae22e860 admin admin] PATH_INFO: `/v3/users/5360b012d34d418a872f222b86c12d93` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:32.358052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d18d7-d033-4ad2-a656-fc56ae22e860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5360b012d34d418a872f222b86c12d93)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:32.359997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d18d7-d033-4ad2-a656-fc56ae22e860 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:32.377718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2327f-ec80-4294-aa08-945de233ed16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:32.378354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2327f-ec80-4294-aa08-945de233ed16 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:32.378626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2327f-ec80-4294-aa08-945de233ed16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:32.378894 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2327f-ec80-4294-aa08-945de233ed16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:32.383510 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-15e2327f-ec80-4294-aa08-945de233ed16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:32.422206 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-280d18d7-d033-4ad2-a656-fc56ae22e860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5360b012d34d418a872f222b86c12d93'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:32.431646 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-280d18d7-d033-4ad2-a656-fc56ae22e860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5360b012d34d418a872f222b86c12d93'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:32.447151 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-280d18d7-d033-4ad2-a656-fc56ae22e860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5360b012d34d418a872f222b86c12d93'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:32.454266 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2327f-ec80-4294-aa08-945de233ed16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:32.455690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2327f-ec80-4294-aa08-945de233ed16 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:32.464579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-280d18d7-d033-4ad2-a656-fc56ae22e860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5360b012d34d418a872f222b86c12d93'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:32.465633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-280d18d7-d033-4ad2-a656-fc56ae22e860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5360b012d34d418a872f222b86c12d93'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:32.467804 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 986/1965] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:32.469233 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bb82b-9853-488b-a9ee-0c9d16072d0a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:32.469524 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bb82b-9853-488b-a9ee-0c9d16072d0a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:32.469799 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bb82b-9853-488b-a9ee-0c9d16072d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:32.473638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 980/1966] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:32 2022] DELETE /identity/v3/users/5360b012d34d418a872f222b86c12d93 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:32.478273 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cbc62-d87f-4afc-8089-57838cb45444 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:32.478645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cbc62-d87f-4afc-8089-57838cb45444 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:32.478899 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cbc62-d87f-4afc-8089-57838cb45444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:32.501242 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f91cbc62-d87f-4afc-8089-57838cb45444 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:32.502657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f91cbc62-d87f-4afc-8089-57838cb45444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:32.545583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-552bb82b-9853-488b-a9ee-0c9d16072d0a None None] MFA Rules not processed for user `bf714e0f43924fa5b1c8cc6f8c5295c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:32.550325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-552bb82b-9853-488b-a9ee-0c9d16072d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:32.591465 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:32.605167 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:32.628144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 981/1967] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:32.638279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cbc62-d87f-4afc-8089-57838cb45444 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:32.639801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f91cbc62-d87f-4afc-8089-57838cb45444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:32.670161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 987/1968] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:32.673770 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bb82b-9853-488b-a9ee-0c9d16072d0a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:32.679217 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-552bb82b-9853-488b-a9ee-0c9d16072d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:32.722566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bb82b-9853-488b-a9ee-0c9d16072d0a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:32.729112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e974dbe-0b91-4319-9ad7-226fa422fa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:32.730313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e974dbe-0b91-4319-9ad7-226fa422fa88 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:32.730884 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e974dbe-0b91-4319-9ad7-226fa422fa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:32.731529 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e974dbe-0b91-4319-9ad7-226fa422fa88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:32.732582 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e974dbe-0b91-4319-9ad7-226fa422fa88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:32.735812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e974dbe-0b91-4319-9ad7-226fa422fa88 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:32.769787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cbc62-d87f-4afc-8089-57838cb45444 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:32.773229 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464e31e-3f9b-4d47-adc2-9d43adb10d28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:32.773925 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464e31e-3f9b-4d47-adc2-9d43adb10d28 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:32.774326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464e31e-3f9b-4d47-adc2-9d43adb10d28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:32.774659 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464e31e-3f9b-4d47-adc2-9d43adb10d28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:32.796899 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464e31e-3f9b-4d47-adc2-9d43adb10d28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:32.798791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464e31e-3f9b-4d47-adc2-9d43adb10d28 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:32.818527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 988/1969] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:07:32 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:32.819663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e974dbe-0b91-4319-9ad7-226fa422fa88 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:32.820299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 982/1970] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:07:32.899185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e974dbe-0b91-4319-9ad7-226fa422fa88 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:32.903984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e37a0-14af-41ba-9569-35b5967fec8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:32.904592 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e37a0-14af-41ba-9569-35b5967fec8e admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:32.904882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e37a0-14af-41ba-9569-35b5967fec8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:32.905129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e37a0-14af-41ba-9569-35b5967fec8e admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:32.905750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e37a0-14af-41ba-9569-35b5967fec8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:32.907709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e37a0-14af-41ba-9569-35b5967fec8e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:32.943416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 989/1971] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:07:32 2022] POST /identity/v3/users => generated 298 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:32.947376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464e31e-3f9b-4d47-adc2-9d43adb10d28 service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:32.961517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464e31e-3f9b-4d47-adc2-9d43adb10d28 service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:32.970240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eae12e-c765-49d8-bcfc-a2850adb007b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:32.972267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eae12e-c765-49d8-bcfc-a2850adb007b admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:32.972556 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eae12e-c765-49d8-bcfc-a2850adb007b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:32.973808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eae12e-c765-49d8-bcfc-a2850adb007b admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:32.974841 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eae12e-c765-49d8-bcfc-a2850adb007b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:32.978577 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eae12e-c765-49d8-bcfc-a2850adb007b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:32.993684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 983/1972] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:07:32.999272 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e37a0-14af-41ba-9569-35b5967fec8e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:33.031381 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e37a0-14af-41ba-9569-35b5967fec8e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:33.042569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465557ef-789a-4e69-a58d-b185ce770af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:33.043259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465557ef-789a-4e69-a58d-b185ce770af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:33.043539 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465557ef-789a-4e69-a58d-b185ce770af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:33.044773 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465557ef-789a-4e69-a58d-b185ce770af8 admin admin] PATH_INFO: `/v3/projects/1a8d1a84eabc42cea8f979d027944109/users/fb8effb6c6e44fbf98a725666a530dce/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:33.077040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465557ef-789a-4e69-a58d-b185ce770af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a8d1a84eabc42cea8f979d027944109, user_id=fb8effb6c6e44fbf98a725666a530dce, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:33.079437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465557ef-789a-4e69-a58d-b185ce770af8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:33.098172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 990/1973] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:32 2022] PUT /identity/v3/projects/1a8d1a84eabc42cea8f979d027944109/users/fb8effb6c6e44fbf98a725666a530dce/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:33.104721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eae12e-c765-49d8-bcfc-a2850adb007b admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:33.123244 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eae12e-c765-49d8-bcfc-a2850adb007b admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:33.197080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a1606-7454-4867-8cfc-f53e96cdc05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:33.197080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a1606-7454-4867-8cfc-f53e96cdc05c admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:33.197080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a1606-7454-4867-8cfc-f53e96cdc05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:33.197080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a1606-7454-4867-8cfc-f53e96cdc05c admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:33.197536 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a1606-7454-4867-8cfc-f53e96cdc05c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:33.197536 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a1606-7454-4867-8cfc-f53e96cdc05c admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:33.206530 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 984/1974] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:33.211308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465557ef-789a-4e69-a58d-b185ce770af8 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:33.240752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465557ef-789a-4e69-a58d-b185ce770af8 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:33.252298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6b96f-eaef-4481-8496-5567f8caab86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:33.253325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6b96f-eaef-4481-8496-5567f8caab86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:33.253727 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6b96f-eaef-4481-8496-5567f8caab86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:33.254149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6b96f-eaef-4481-8496-5567f8caab86 admin admin] PATH_INFO: `/v3/projects/1a8d1a84eabc42cea8f979d027944109/users/fb8effb6c6e44fbf98a725666a530dce/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:33.256402 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6b96f-eaef-4481-8496-5567f8caab86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a8d1a84eabc42cea8f979d027944109, user_id=fb8effb6c6e44fbf98a725666a530dce, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:33.260111 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6b96f-eaef-4481-8496-5567f8caab86 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:33.272131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 991/1975] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:33 2022] PUT /identity/v3/projects/1a8d1a84eabc42cea8f979d027944109/users/fb8effb6c6e44fbf98a725666a530dce/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:33.282702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc1bc3-4a8a-4f29-b8ec-b53cfdcb791c None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:33.282992 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc1bc3-4a8a-4f29-b8ec-b53cfdcb791c None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:33.283268 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc1bc3-4a8a-4f29-b8ec-b53cfdcb791c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:33.319665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-aedc1bc3-4a8a-4f29-b8ec-b53cfdcb791c None None] MFA Rules not processed for user `fb8effb6c6e44fbf98a725666a530dce`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:33.320968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-aedc1bc3-4a8a-4f29-b8ec-b53cfdcb791c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:33.347194 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6b96f-eaef-4481-8496-5567f8caab86 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:33.349492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-14c6b96f-eaef-4481-8496-5567f8caab86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:33.470842 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6b96f-eaef-4481-8496-5567f8caab86 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:33.471086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 985/1976] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:33.471086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc1bc3-4a8a-4f29-b8ec-b53cfdcb791c None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:33.476435 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-aedc1bc3-4a8a-4f29-b8ec-b53cfdcb791c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:33.477504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160ed46-a7dc-4285-9a85-7a0ce4ace28e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:33.478715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160ed46-a7dc-4285-9a85-7a0ce4ace28e service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:33.478715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160ed46-a7dc-4285-9a85-7a0ce4ace28e service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:33.478931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160ed46-a7dc-4285-9a85-7a0ce4ace28e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:33.483136 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e160ed46-a7dc-4285-9a85-7a0ce4ace28e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:33.492342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:33.501897 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:33.600017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160ed46-a7dc-4285-9a85-7a0ce4ace28e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:33.602801 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160ed46-a7dc-4285-9a85-7a0ce4ace28e service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:33.603336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc1bc3-4a8a-4f29-b8ec-b53cfdcb791c None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:33.614674 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 992/1977] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:07:33.615485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160ed46-a7dc-4285-9a85-7a0ce4ace28e service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:33.617705 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e160ed46-a7dc-4285-9a85-7a0ce4ace28e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:33.620309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702de1a0-0abb-42f6-8f73-4ecdb4462127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:33.621188 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702de1a0-0abb-42f6-8f73-4ecdb4462127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:33.621470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702de1a0-0abb-42f6-8f73-4ecdb4462127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:33.621744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702de1a0-0abb-42f6-8f73-4ecdb4462127 admin admin] PATH_INFO: `/v3/projects/56d9a95d2dc94d67953c8b87f374bc2a` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:33.624245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702de1a0-0abb-42f6-8f73-4ecdb4462127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56d9a95d2dc94d67953c8b87f374bc2a)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:33.626223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702de1a0-0abb-42f6-8f73-4ecdb4462127 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:33.634501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160ed46-a7dc-4285-9a85-7a0ce4ace28e service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:33.638993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e0209-5dae-4b2f-ba80-50017091794f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:33.639730 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e0209-5dae-4b2f-ba80-50017091794f service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:33.640367 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e0209-5dae-4b2f-ba80-50017091794f service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:33.641177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e0209-5dae-4b2f-ba80-50017091794f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:33.655946 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e0209-5dae-4b2f-ba80-50017091794f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:33.657913 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e0209-5dae-4b2f-ba80-50017091794f service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:33.672037 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 993/1978] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:33.710232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-702de1a0-0abb-42f6-8f73-4ecdb4462127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56d9a95d2dc94d67953c8b87f374bc2a', 'request_id': 'req-702de1a0-0abb-42f6-8f73-4ecdb4462127'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:33.725710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-702de1a0-0abb-42f6-8f73-4ecdb4462127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56d9a95d2dc94d67953c8b87f374bc2a', 'request_id': 'req-702de1a0-0abb-42f6-8f73-4ecdb4462127'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:33.742987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-702de1a0-0abb-42f6-8f73-4ecdb4462127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56d9a95d2dc94d67953c8b87f374bc2a', 'request_id': 'req-702de1a0-0abb-42f6-8f73-4ecdb4462127'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:33.762291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-702de1a0-0abb-42f6-8f73-4ecdb4462127 admin admin] The token cache is being invalidate because project 56d9a95d2dc94d67953c8b87f374bc2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:33.762858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-702de1a0-0abb-42f6-8f73-4ecdb4462127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:33.763837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-702de1a0-0abb-42f6-8f73-4ecdb4462127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:33.766957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 986/1979] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:33 2022] DELETE /identity/v3/projects/56d9a95d2dc94d67953c8b87f374bc2a => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:33.770154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e0209-5dae-4b2f-ba80-50017091794f service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:33.791799 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e0209-5dae-4b2f-ba80-50017091794f service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:33.812248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702de1a0-0abb-42f6-8f73-4ecdb4462127 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:33.812894 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-702de1a0-0abb-42f6-8f73-4ecdb4462127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:33.872663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7678e38-9621-4495-98cb-bfe806670eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:33.873572 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7678e38-9621-4495-98cb-bfe806670eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:33.873910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7678e38-9621-4495-98cb-bfe806670eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:33.874363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7678e38-9621-4495-98cb-bfe806670eb5 admin admin] PATH_INFO: `/v3/users/164705b7614a42cb81e4284512f3832a` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:33.876459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7678e38-9621-4495-98cb-bfe806670eb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=164705b7614a42cb81e4284512f3832a)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:33.879475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7678e38-9621-4495-98cb-bfe806670eb5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:33.888112 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702de1a0-0abb-42f6-8f73-4ecdb4462127 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:33.893193 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d7bc8-2e22-42bc-b8bc-b30efb45f16c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:33.894531 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d7bc8-2e22-42bc-b8bc-b30efb45f16c service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:33.894845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d7bc8-2e22-42bc-b8bc-b30efb45f16c service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:33.895088 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d7bc8-2e22-42bc-b8bc-b30efb45f16c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:33.897805 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d15d7bc8-2e22-42bc-b8bc-b30efb45f16c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:33.937201 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f7678e38-9621-4495-98cb-bfe806670eb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '164705b7614a42cb81e4284512f3832a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:33.938283 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f7678e38-9621-4495-98cb-bfe806670eb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '164705b7614a42cb81e4284512f3832a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:33.947068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f7678e38-9621-4495-98cb-bfe806670eb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '164705b7614a42cb81e4284512f3832a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:33.955778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f7678e38-9621-4495-98cb-bfe806670eb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '164705b7614a42cb81e4284512f3832a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:33.965709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:33.969431 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d7bc8-2e22-42bc-b8bc-b30efb45f16c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:33.971716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d7bc8-2e22-42bc-b8bc-b30efb45f16c service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:33.972938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f7678e38-9621-4495-98cb-bfe806670eb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '164705b7614a42cb81e4284512f3832a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:33.984492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 987/1980] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:07:33.989791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 994/1981] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:33 2022] DELETE /identity/v3/users/164705b7614a42cb81e4284512f3832a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:34.323022 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d7bc8-2e22-42bc-b8bc-b30efb45f16c service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:34.324685 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d15d7bc8-2e22-42bc-b8bc-b30efb45f16c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:34.369255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d7bc8-2e22-42bc-b8bc-b30efb45f16c service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:34.463232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39efe83-c507-4bd0-b57e-d30fc9444ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:34.464067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39efe83-c507-4bd0-b57e-d30fc9444ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:34.464347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39efe83-c507-4bd0-b57e-d30fc9444ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:34.464682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39efe83-c507-4bd0-b57e-d30fc9444ec0 admin admin] PATH_INFO: `/v3/projects/f12dc53b909b46bfb1c882b3f86722c7` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:34.465948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39efe83-c507-4bd0-b57e-d30fc9444ec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f12dc53b909b46bfb1c882b3f86722c7)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:34.467993 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39efe83-c507-4bd0-b57e-d30fc9444ec0 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:34.527750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d39efe83-c507-4bd0-b57e-d30fc9444ec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f12dc53b909b46bfb1c882b3f86722c7', 'request_id': 'req-d39efe83-c507-4bd0-b57e-d30fc9444ec0'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:34.539945 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d39efe83-c507-4bd0-b57e-d30fc9444ec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f12dc53b909b46bfb1c882b3f86722c7', 'request_id': 'req-d39efe83-c507-4bd0-b57e-d30fc9444ec0'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:34.547797 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d39efe83-c507-4bd0-b57e-d30fc9444ec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f12dc53b909b46bfb1c882b3f86722c7', 'request_id': 'req-d39efe83-c507-4bd0-b57e-d30fc9444ec0'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:34.560446 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d39efe83-c507-4bd0-b57e-d30fc9444ec0 admin admin] The token cache is being invalidate because project f12dc53b909b46bfb1c882b3f86722c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:34.560736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d39efe83-c507-4bd0-b57e-d30fc9444ec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:34.561589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d39efe83-c507-4bd0-b57e-d30fc9444ec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:34.563136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 988/1982] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:34 2022] DELETE /identity/v3/projects/f12dc53b909b46bfb1c882b3f86722c7 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:34.568074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a120eb-2e64-4dff-9e14-ad79cee1dd0d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:34.568360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a120eb-2e64-4dff-9e14-ad79cee1dd0d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:34.568637 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a120eb-2e64-4dff-9e14-ad79cee1dd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:34.590630 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-89a120eb-2e64-4dff-9e14-ad79cee1dd0d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:34.591845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-89a120eb-2e64-4dff-9e14-ad79cee1dd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:34.597769 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:34.665782 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 995/1983] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:34.672702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8e56b-c28d-43fc-b0ff-a3397583ae43 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:34.672990 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8e56b-c28d-43fc-b0ff-a3397583ae43 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:34.673266 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8e56b-c28d-43fc-b0ff-a3397583ae43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:34.696337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-74c8e56b-c28d-43fc-b0ff-a3397583ae43 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:34.697719 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-74c8e56b-c28d-43fc-b0ff-a3397583ae43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:34.708982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 989/1984] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:34.713188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a120eb-2e64-4dff-9e14-ad79cee1dd0d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:34.723854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a120eb-2e64-4dff-9e14-ad79cee1dd0d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:34.728130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf64276-0abb-44e2-8f93-6add99ae7161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:34.728745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf64276-0abb-44e2-8f93-6add99ae7161 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:34.729019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf64276-0abb-44e2-8f93-6add99ae7161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:34.729286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf64276-0abb-44e2-8f93-6add99ae7161 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:34.729929 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf64276-0abb-44e2-8f93-6add99ae7161 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:07:34.730257 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf64276-0abb-44e2-8f93-6add99ae7161 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:34.731842 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf64276-0abb-44e2-8f93-6add99ae7161 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:34.745710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 996/1985] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:07:34.749961 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8e56b-c28d-43fc-b0ff-a3397583ae43 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:34.762391 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8e56b-c28d-43fc-b0ff-a3397583ae43 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:34.767117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981564ab-810f-4448-a531-e4257b09e51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:34.767739 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981564ab-810f-4448-a531-e4257b09e51f admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:34.768018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981564ab-810f-4448-a531-e4257b09e51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:34.768290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981564ab-810f-4448-a531-e4257b09e51f admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:34.769024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981564ab-810f-4448-a531-e4257b09e51f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:34.771039 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981564ab-810f-4448-a531-e4257b09e51f admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:34.803453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 990/1986] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:07:34 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:34.809189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf64276-0abb-44e2-8f93-6add99ae7161 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:34.820072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf64276-0abb-44e2-8f93-6add99ae7161 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:34.895785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0abe5-df5f-4673-8f5d-d626e3043c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:34.896708 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0abe5-df5f-4673-8f5d-d626e3043c12 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:34.897131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0abe5-df5f-4673-8f5d-d626e3043c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:34.898660 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0abe5-df5f-4673-8f5d-d626e3043c12 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:34.898774 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0abe5-df5f-4673-8f5d-d626e3043c12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:34.900730 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0abe5-df5f-4673-8f5d-d626e3043c12 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:34.932004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 997/1987] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:07:34 2022] POST /identity/v3/users => generated 285 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:07:34.932585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981564ab-810f-4448-a531-e4257b09e51f admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:34.943895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981564ab-810f-4448-a531-e4257b09e51f admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:34.951304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16cbcc-5714-437a-ab43-220cfeaefce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:34.951592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16cbcc-5714-437a-ab43-220cfeaefce6 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:34.951859 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16cbcc-5714-437a-ab43-220cfeaefce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:34.952102 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16cbcc-5714-437a-ab43-220cfeaefce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:34.953282 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16cbcc-5714-437a-ab43-220cfeaefce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:34.955174 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16cbcc-5714-437a-ab43-220cfeaefce6 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:34.967179 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 991/1988] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:34.971888 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0abe5-df5f-4673-8f5d-d626e3043c12 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:34.991884 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0abe5-df5f-4673-8f5d-d626e3043c12 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:35.003276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5230e82-d331-4f9e-8664-f5d51657be89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:35.004364 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5230e82-d331-4f9e-8664-f5d51657be89 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:35.005428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5230e82-d331-4f9e-8664-f5d51657be89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:35.005713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5230e82-d331-4f9e-8664-f5d51657be89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:35.006682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5230e82-d331-4f9e-8664-f5d51657be89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:35.009383 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5230e82-d331-4f9e-8664-f5d51657be89 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:35.027047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 998/1989] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:35.028543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16cbcc-5714-437a-ab43-220cfeaefce6 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:35.040597 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16cbcc-5714-437a-ab43-220cfeaefce6 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:35.045652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d550ba-a8ce-420f-b802-e2c66fb611e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:35.046357 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d550ba-a8ce-420f-b802-e2c66fb611e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:35.046638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d550ba-a8ce-420f-b802-e2c66fb611e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:35.046911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d550ba-a8ce-420f-b802-e2c66fb611e7 admin admin] PATH_INFO: `/v3/projects/89e482ad43ce4cd6ab730b764f879954/users/a1f5bc3f97ff4d2a85e6d670a3eb9b7d/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:35.064561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d550ba-a8ce-420f-b802-e2c66fb611e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e482ad43ce4cd6ab730b764f879954, user_id=a1f5bc3f97ff4d2a85e6d670a3eb9b7d, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:35.067524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d550ba-a8ce-420f-b802-e2c66fb611e7 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:35.098027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 992/1990] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:35 2022] PUT /identity/v3/projects/89e482ad43ce4cd6ab730b764f879954/users/a1f5bc3f97ff4d2a85e6d670a3eb9b7d/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:35.110029 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43547b2f-de6e-4e3b-9416-dc8d1f5775ba None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:35.110404 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43547b2f-de6e-4e3b-9416-dc8d1f5775ba None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:35.110743 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43547b2f-de6e-4e3b-9416-dc8d1f5775ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:35.145330 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-43547b2f-de6e-4e3b-9416-dc8d1f5775ba None None] MFA Rules not processed for user `a1f5bc3f97ff4d2a85e6d670a3eb9b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:35.147297 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-43547b2f-de6e-4e3b-9416-dc8d1f5775ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:35.242672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 999/1991] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:35.247412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d550ba-a8ce-420f-b802-e2c66fb611e7 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:35.269601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:35.317773 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d550ba-a8ce-420f-b802-e2c66fb611e7 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:35.322735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40218f62-1785-4e8d-88bd-105f68f707f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:35.323345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40218f62-1785-4e8d-88bd-105f68f707f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:35.323631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40218f62-1785-4e8d-88bd-105f68f707f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:35.323954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40218f62-1785-4e8d-88bd-105f68f707f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:35.324605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40218f62-1785-4e8d-88bd-105f68f707f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:35.326583 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40218f62-1785-4e8d-88bd-105f68f707f7 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:35.355703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 993/1992] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:07:35 2022] POST /identity/v3/projects => generated 369 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:35.359870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43547b2f-de6e-4e3b-9416-dc8d1f5775ba None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:35.371106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43547b2f-de6e-4e3b-9416-dc8d1f5775ba None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:35.374880 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:35.435283 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece91cd-fe84-4966-9865-df589da8419b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:35.435899 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece91cd-fe84-4966-9865-df589da8419b admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:35.436183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece91cd-fe84-4966-9865-df589da8419b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:35.436456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece91cd-fe84-4966-9865-df589da8419b admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:35.437100 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece91cd-fe84-4966-9865-df589da8419b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:35.439373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece91cd-fe84-4966-9865-df589da8419b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:35.466517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1000/1993] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:07:35 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:07:35.467800 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40218f62-1785-4e8d-88bd-105f68f707f7 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:35.479004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40218f62-1785-4e8d-88bd-105f68f707f7 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:35.484293 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d0984-6d0c-41b5-82d2-23e742e3c44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:35.484895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d0984-6d0c-41b5-82d2-23e742e3c44c admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:35.485170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d0984-6d0c-41b5-82d2-23e742e3c44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:35.485728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d0984-6d0c-41b5-82d2-23e742e3c44c admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:35.486402 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d0984-6d0c-41b5-82d2-23e742e3c44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:35.488034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d0984-6d0c-41b5-82d2-23e742e3c44c admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:35.501644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 994/1994] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:35.505995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece91cd-fe84-4966-9865-df589da8419b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:35.516797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece91cd-fe84-4966-9865-df589da8419b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:35.521512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638767e-e5d9-4042-bda5-c6670e64abf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:35.522207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638767e-e5d9-4042-bda5-c6670e64abf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:35.522489 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638767e-e5d9-4042-bda5-c6670e64abf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:35.522770 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638767e-e5d9-4042-bda5-c6670e64abf3 admin admin] PATH_INFO: `/v3/projects/ac3fa915e6d1494a9a5012736272f83e/users/b823bcbfa2734bb1a8b86b4a490d4a49/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:35.537840 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638767e-e5d9-4042-bda5-c6670e64abf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3fa915e6d1494a9a5012736272f83e, user_id=b823bcbfa2734bb1a8b86b4a490d4a49, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:35.540588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638767e-e5d9-4042-bda5-c6670e64abf3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:35.558178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1001/1995] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:35 2022] PUT /identity/v3/projects/ac3fa915e6d1494a9a5012736272f83e/users/b823bcbfa2734bb1a8b86b4a490d4a49/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:35.562218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d0984-6d0c-41b5-82d2-23e742e3c44c admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:35.576971 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d0984-6d0c-41b5-82d2-23e742e3c44c admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:35.647195 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146ee85-6d3e-4bdc-8d73-6e22c3a93ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:35.647836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146ee85-6d3e-4bdc-8d73-6e22c3a93ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:35.648096 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146ee85-6d3e-4bdc-8d73-6e22c3a93ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:35.648629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146ee85-6d3e-4bdc-8d73-6e22c3a93ff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:35.649045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146ee85-6d3e-4bdc-8d73-6e22c3a93ff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:35.651245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146ee85-6d3e-4bdc-8d73-6e22c3a93ff5 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:35.661658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 995/1996] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:35.666615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638767e-e5d9-4042-bda5-c6670e64abf3 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:35.682893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638767e-e5d9-4042-bda5-c6670e64abf3 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:35.688222 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e2438-f79a-4e03-9533-ed966a02482e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:35.688923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e2438-f79a-4e03-9533-ed966a02482e admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:35.689203 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e2438-f79a-4e03-9533-ed966a02482e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:35.689474 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e2438-f79a-4e03-9533-ed966a02482e admin admin] PATH_INFO: `/v3/projects/ac3fa915e6d1494a9a5012736272f83e/users/b823bcbfa2734bb1a8b86b4a490d4a49/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:35.691445 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e2438-f79a-4e03-9533-ed966a02482e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3fa915e6d1494a9a5012736272f83e, user_id=b823bcbfa2734bb1a8b86b4a490d4a49, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:35.694073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e2438-f79a-4e03-9533-ed966a02482e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:35.705477 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1002/1997] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:35 2022] PUT /identity/v3/projects/ac3fa915e6d1494a9a5012736272f83e/users/b823bcbfa2734bb1a8b86b4a490d4a49/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:07:35.715636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d789b-6749-4bf1-8b2c-90ce598f3f43 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:35.715987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d789b-6749-4bf1-8b2c-90ce598f3f43 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:35.716348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d789b-6749-4bf1-8b2c-90ce598f3f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:35.747733 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-c19d789b-6749-4bf1-8b2c-90ce598f3f43 None None] MFA Rules not processed for user `b823bcbfa2734bb1a8b86b4a490d4a49`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:35.748961 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-c19d789b-6749-4bf1-8b2c-90ce598f3f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:35.836062 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 996/1998] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:07:35.848219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e2438-f79a-4e03-9533-ed966a02482e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:35.851484 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-443e2438-f79a-4e03-9533-ed966a02482e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:35.928511 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e2438-f79a-4e03-9533-ed966a02482e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:35.933165 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db6159-b0f0-46fa-ab88-e9d1e065983f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:35.933925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db6159-b0f0-46fa-ab88-e9d1e065983f service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:35.934313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db6159-b0f0-46fa-ab88-e9d1e065983f service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:35.934682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db6159-b0f0-46fa-ab88-e9d1e065983f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:35.945198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db6159-b0f0-46fa-ab88-e9d1e065983f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:35.946685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db6159-b0f0-46fa-ab88-e9d1e065983f service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:35.957253 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1003/1999] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:37.752060 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d789b-6749-4bf1-8b2c-90ce598f3f43 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:37.769918 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:37.826054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d789b-6749-4bf1-8b2c-90ce598f3f43 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:37.834304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27966c6c-04ec-4266-a52a-cb06eb261ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:37.834304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27966c6c-04ec-4266-a52a-cb06eb261ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:37.834988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27966c6c-04ec-4266-a52a-cb06eb261ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:37.834988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27966c6c-04ec-4266-a52a-cb06eb261ee8 admin admin] PATH_INFO: `/v3/users/a1f5bc3f97ff4d2a85e6d670a3eb9b7d` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:37.838169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27966c6c-04ec-4266-a52a-cb06eb261ee8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1f5bc3f97ff4d2a85e6d670a3eb9b7d)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:37.840245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27966c6c-04ec-4266-a52a-cb06eb261ee8 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:37.861473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db6159-b0f0-46fa-ab88-e9d1e065983f service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:37.863161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d4db6159-b0f0-46fa-ab88-e9d1e065983f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:37.883764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db6159-b0f0-46fa-ab88-e9d1e065983f service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:37.893172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8188e-9b57-4672-b71e-40759465fb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:37.894355 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8188e-9b57-4672-b71e-40759465fb71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:37.894849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8188e-9b57-4672-b71e-40759465fb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:37.895334 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8188e-9b57-4672-b71e-40759465fb71 admin admin] PATH_INFO: `/v3/users/196df48558094813af62bc26ac2c4a91` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:37.896676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8188e-9b57-4672-b71e-40759465fb71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=196df48558094813af62bc26ac2c4a91)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:37.899211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8188e-9b57-4672-b71e-40759465fb71 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:37.901223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-27966c6c-04ec-4266-a52a-cb06eb261ee8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1f5bc3f97ff4d2a85e6d670a3eb9b7d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:37.909589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-27966c6c-04ec-4266-a52a-cb06eb261ee8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1f5bc3f97ff4d2a85e6d670a3eb9b7d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:37.923805 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-27966c6c-04ec-4266-a52a-cb06eb261ee8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1f5bc3f97ff4d2a85e6d670a3eb9b7d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:37.933100 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-27966c6c-04ec-4266-a52a-cb06eb261ee8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1f5bc3f97ff4d2a85e6d670a3eb9b7d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:37.933831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-27966c6c-04ec-4266-a52a-cb06eb261ee8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1f5bc3f97ff4d2a85e6d670a3eb9b7d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:37.939988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 997/2000] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:37 2022] DELETE /identity/v3/users/a1f5bc3f97ff4d2a85e6d670a3eb9b7d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:37.940988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27966c6c-04ec-4266-a52a-cb06eb261ee8 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:37.942366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-27966c6c-04ec-4266-a52a-cb06eb261ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:37.962619 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-20a8188e-9b57-4672-b71e-40759465fb71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '196df48558094813af62bc26ac2c4a91'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:37.963298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-20a8188e-9b57-4672-b71e-40759465fb71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '196df48558094813af62bc26ac2c4a91'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:37.972446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-20a8188e-9b57-4672-b71e-40759465fb71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '196df48558094813af62bc26ac2c4a91'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:37.974269 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:37.983879 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-20a8188e-9b57-4672-b71e-40759465fb71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '196df48558094813af62bc26ac2c4a91'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:38.003578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-20a8188e-9b57-4672-b71e-40759465fb71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '196df48558094813af62bc26ac2c4a91'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:38.011424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1004/2001] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:37 2022] DELETE /identity/v3/users/196df48558094813af62bc26ac2c4a91 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:38.013849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4151ce-5382-4541-a8c6-75a0cac4f91c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:38.014147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4151ce-5382-4541-a8c6-75a0cac4f91c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:38.014442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4151ce-5382-4541-a8c6-75a0cac4f91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:38.018781 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27966c6c-04ec-4266-a52a-cb06eb261ee8 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:38.035072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7b4151ce-5382-4541-a8c6-75a0cac4f91c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:38.036953 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7b4151ce-5382-4541-a8c6-75a0cac4f91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:38.073045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:38.092236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a333ca-9029-41ab-af70-27c947452c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:38.092868 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a333ca-9029-41ab-af70-27c947452c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:38.093144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a333ca-9029-41ab-af70-27c947452c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:38.093411 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a333ca-9029-41ab-af70-27c947452c66 admin admin] PATH_INFO: `/v3/users/bf714e0f43924fa5b1c8cc6f8c5295c8` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:38.094582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a333ca-9029-41ab-af70-27c947452c66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf714e0f43924fa5b1c8cc6f8c5295c8)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:38.096484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a333ca-9029-41ab-af70-27c947452c66 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:38.102549 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1005/2002] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:38.103810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bddb8-f6e7-458a-9b90-98ece3960a9e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:38.104083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bddb8-f6e7-458a-9b90-98ece3960a9e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:38.104350 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bddb8-f6e7-458a-9b90-98ece3960a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:38.126573 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4d8bddb8-f6e7-458a-9b90-98ece3960a9e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:38.127981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4d8bddb8-f6e7-458a-9b90-98ece3960a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:38.136654 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1006/2003] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:38.137501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8bddb8-f6e7-458a-9b90-98ece3960a9e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:38.138840 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4d8bddb8-f6e7-458a-9b90-98ece3960a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:38.161762 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-33a333ca-9029-41ab-af70-27c947452c66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf714e0f43924fa5b1c8cc6f8c5295c8'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:38.171539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-33a333ca-9029-41ab-af70-27c947452c66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf714e0f43924fa5b1c8cc6f8c5295c8'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:38.185666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-33a333ca-9029-41ab-af70-27c947452c66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf714e0f43924fa5b1c8cc6f8c5295c8'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:38.193014 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-33a333ca-9029-41ab-af70-27c947452c66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf714e0f43924fa5b1c8cc6f8c5295c8'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:38.193855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-33a333ca-9029-41ab-af70-27c947452c66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf714e0f43924fa5b1c8cc6f8c5295c8'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:38.202304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 998/2004] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:37 2022] DELETE /identity/v3/users/bf714e0f43924fa5b1c8cc6f8c5295c8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:38.202304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a333ca-9029-41ab-af70-27c947452c66 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:38.202560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-33a333ca-9029-41ab-af70-27c947452c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:38.208968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8bddb8-f6e7-458a-9b90-98ece3960a9e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:38.289202 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a333ca-9029-41ab-af70-27c947452c66 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:38.289391 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d826e5-b7d8-4fe9-95c8-4a767b604227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:38.289845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d826e5-b7d8-4fe9-95c8-4a767b604227 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:38.290180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d826e5-b7d8-4fe9-95c8-4a767b604227 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:38.290424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d826e5-b7d8-4fe9-95c8-4a767b604227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:38.292421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-11d826e5-b7d8-4fe9-95c8-4a767b604227 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:38.293984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1a100-aee1-4ee5-9be6-864195dc8b94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:38.294632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1a100-aee1-4ee5-9be6-864195dc8b94 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:38.294915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1a100-aee1-4ee5-9be6-864195dc8b94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:38.295195 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1a100-aee1-4ee5-9be6-864195dc8b94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:38.297125 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e0c1a100-aee1-4ee5-9be6-864195dc8b94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:38.338671 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:38.370644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1a100-aee1-4ee5-9be6-864195dc8b94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:38.370822 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d826e5-b7d8-4fe9-95c8-4a767b604227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:38.372261 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d826e5-b7d8-4fe9-95c8-4a767b604227 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:38.372369 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1a100-aee1-4ee5-9be6-864195dc8b94 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:38.385701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1007/2005] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:38.387123 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95331421-ad29-489d-b288-8afcc75f8ea3 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:38.387460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95331421-ad29-489d-b288-8afcc75f8ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:38.387689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95331421-ad29-489d-b288-8afcc75f8ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:38.388549 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 999/2006] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:38.410016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-95331421-ad29-489d-b288-8afcc75f8ea3 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:38.411714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-95331421-ad29-489d-b288-8afcc75f8ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:38.420579 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1008/2007] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:38.432572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1a100-aee1-4ee5-9be6-864195dc8b94 service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:38.445460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1a100-aee1-4ee5-9be6-864195dc8b94 service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:38.452983 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676d8d6-e4f5-4de4-8912-264a4d6730e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:38.453813 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676d8d6-e4f5-4de4-8912-264a4d6730e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:38.454208 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676d8d6-e4f5-4de4-8912-264a4d6730e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:38.454543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676d8d6-e4f5-4de4-8912-264a4d6730e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:38.472344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676d8d6-e4f5-4de4-8912-264a4d6730e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:38.474163 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676d8d6-e4f5-4de4-8912-264a4d6730e5 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:38.479894 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:38.489042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1000/2008] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:38.777390 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95331421-ad29-489d-b288-8afcc75f8ea3 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:38.779043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-95331421-ad29-489d-b288-8afcc75f8ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:38.812040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95331421-ad29-489d-b288-8afcc75f8ea3 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:38.812040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35e107-e474-47f6-8c90-9fa5454898cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:38.812040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35e107-e474-47f6-8c90-9fa5454898cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:38.812040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35e107-e474-47f6-8c90-9fa5454898cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:38.812523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35e107-e474-47f6-8c90-9fa5454898cd admin admin] PATH_INFO: `/v3/projects/89e482ad43ce4cd6ab730b764f879954` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:38.812523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35e107-e474-47f6-8c90-9fa5454898cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=89e482ad43ce4cd6ab730b764f879954)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:38.812523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35e107-e474-47f6-8c90-9fa5454898cd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:38.876654 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bd35e107-e474-47f6-8c90-9fa5454898cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89e482ad43ce4cd6ab730b764f879954', 'request_id': 'req-bd35e107-e474-47f6-8c90-9fa5454898cd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:38.912065 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bd35e107-e474-47f6-8c90-9fa5454898cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89e482ad43ce4cd6ab730b764f879954', 'request_id': 'req-bd35e107-e474-47f6-8c90-9fa5454898cd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:38.921408 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bd35e107-e474-47f6-8c90-9fa5454898cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89e482ad43ce4cd6ab730b764f879954', 'request_id': 'req-bd35e107-e474-47f6-8c90-9fa5454898cd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:38.937093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bd35e107-e474-47f6-8c90-9fa5454898cd admin admin] The token cache is being invalidate because project 89e482ad43ce4cd6ab730b764f879954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:38.937387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bd35e107-e474-47f6-8c90-9fa5454898cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:38.938332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bd35e107-e474-47f6-8c90-9fa5454898cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:38.939832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1009/2009] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:38 2022] DELETE /identity/v3/projects/89e482ad43ce4cd6ab730b764f879954 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:38.944000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676d8d6-e4f5-4de4-8912-264a4d6730e5 service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:38.957569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676d8d6-e4f5-4de4-8912-264a4d6730e5 service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:39.035268 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a52d77-b441-47aa-8ef9-e7221d438c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:39.036025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a52d77-b441-47aa-8ef9-e7221d438c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:39.036305 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a52d77-b441-47aa-8ef9-e7221d438c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:39.036580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a52d77-b441-47aa-8ef9-e7221d438c7e admin admin] PATH_INFO: `/v3/users/b823bcbfa2734bb1a8b86b4a490d4a49` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:39.038093 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a52d77-b441-47aa-8ef9-e7221d438c7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b823bcbfa2734bb1a8b86b4a490d4a49)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:39.040973 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a52d77-b441-47aa-8ef9-e7221d438c7e admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:39.061600 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35e107-e474-47f6-8c90-9fa5454898cd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:39.063152 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bd35e107-e474-47f6-8c90-9fa5454898cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:39.078407 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35e107-e474-47f6-8c90-9fa5454898cd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:39.087110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:39.088161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:39.088703 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:39.089216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5 admin admin] PATH_INFO: `/v3/projects/49da20621aec468398e6e1ad654e62bb` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:39.090733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49da20621aec468398e6e1ad654e62bb)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:39.093344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:39.132475 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b9a52d77-b441-47aa-8ef9-e7221d438c7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b823bcbfa2734bb1a8b86b4a490d4a49'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.143982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b9a52d77-b441-47aa-8ef9-e7221d438c7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b823bcbfa2734bb1a8b86b4a490d4a49'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.150941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49da20621aec468398e6e1ad654e62bb', 'request_id': 'req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.160594 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49da20621aec468398e6e1ad654e62bb', 'request_id': 'req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.162479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b9a52d77-b441-47aa-8ef9-e7221d438c7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b823bcbfa2734bb1a8b86b4a490d4a49'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.168253 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49da20621aec468398e6e1ad654e62bb', 'request_id': 'req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.171903 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b9a52d77-b441-47aa-8ef9-e7221d438c7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b823bcbfa2734bb1a8b86b4a490d4a49'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.172838 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b9a52d77-b441-47aa-8ef9-e7221d438c7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b823bcbfa2734bb1a8b86b4a490d4a49'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.181334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1001/2010] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:38 2022] DELETE /identity/v3/users/b823bcbfa2734bb1a8b86b4a490d4a49 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:07:39.184902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5 admin admin] The token cache is being invalidate because project 49da20621aec468398e6e1ad654e62bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:39.185298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.186265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.189324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1010/2011] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:39 2022] DELETE /identity/v3/projects/49da20621aec468398e6e1ad654e62bb => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:39.198367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a52d77-b441-47aa-8ef9-e7221d438c7e admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:39.199819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b9a52d77-b441-47aa-8ef9-e7221d438c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:39.218949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:39.221202 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:39.263316 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:39.303008 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a52d77-b441-47aa-8ef9-e7221d438c7e admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:39.308379 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22427e19-73d3-4f84-a246-e759fc4ab7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:39.309142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22427e19-73d3-4f84-a246-e759fc4ab7a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:39.309466 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f59e49-816f-43d5-bd5c-3eb5d54f96e5 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:39.309572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22427e19-73d3-4f84-a246-e759fc4ab7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:39.309938 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22427e19-73d3-4f84-a246-e759fc4ab7a4 admin admin] PATH_INFO: `/v3/users/6e79f5549c1246e0becb76f7bbc24f91` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:39.311289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22427e19-73d3-4f84-a246-e759fc4ab7a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e79f5549c1246e0becb76f7bbc24f91)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:39.313595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22427e19-73d3-4f84-a246-e759fc4ab7a4 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:39.315591 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12623020-7489-4372-bbe8-1248f501b8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:39.316468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12623020-7489-4372-bbe8-1248f501b8ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:39.316721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12623020-7489-4372-bbe8-1248f501b8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:39.317079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12623020-7489-4372-bbe8-1248f501b8ad admin admin] PATH_INFO: `/v3/projects/e6bc2bbe33934b53ae1c6d15432fa2ea` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:39.318787 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12623020-7489-4372-bbe8-1248f501b8ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6bc2bbe33934b53ae1c6d15432fa2ea)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:39.320871 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12623020-7489-4372-bbe8-1248f501b8ad admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:39.337670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:39.370355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-22427e19-73d3-4f84-a246-e759fc4ab7a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e79f5549c1246e0becb76f7bbc24f91'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.379974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-22427e19-73d3-4f84-a246-e759fc4ab7a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e79f5549c1246e0becb76f7bbc24f91'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.390334 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-12623020-7489-4372-bbe8-1248f501b8ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6bc2bbe33934b53ae1c6d15432fa2ea', 'request_id': 'req-12623020-7489-4372-bbe8-1248f501b8ad'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.396011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-22427e19-73d3-4f84-a246-e759fc4ab7a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e79f5549c1246e0becb76f7bbc24f91'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.400365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-12623020-7489-4372-bbe8-1248f501b8ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6bc2bbe33934b53ae1c6d15432fa2ea', 'request_id': 'req-12623020-7489-4372-bbe8-1248f501b8ad'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.405346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-22427e19-73d3-4f84-a246-e759fc4ab7a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e79f5549c1246e0becb76f7bbc24f91'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.406270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-22427e19-73d3-4f84-a246-e759fc4ab7a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e79f5549c1246e0becb76f7bbc24f91'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.407338 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-12623020-7489-4372-bbe8-1248f501b8ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6bc2bbe33934b53ae1c6d15432fa2ea', 'request_id': 'req-12623020-7489-4372-bbe8-1248f501b8ad'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.413191 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1002/2012] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:39 2022] DELETE /identity/v3/users/6e79f5549c1246e0becb76f7bbc24f91 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:39.421062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-12623020-7489-4372-bbe8-1248f501b8ad admin admin] The token cache is being invalidate because project e6bc2bbe33934b53ae1c6d15432fa2ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:39.421475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-12623020-7489-4372-bbe8-1248f501b8ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.422279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-12623020-7489-4372-bbe8-1248f501b8ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.426173 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1011/2013] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:39 2022] DELETE /identity/v3/projects/e6bc2bbe33934b53ae1c6d15432fa2ea => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:39.431232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22427e19-73d3-4f84-a246-e759fc4ab7a4 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:39.432787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-22427e19-73d3-4f84-a246-e759fc4ab7a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:39.510019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12623020-7489-4372-bbe8-1248f501b8ad admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:39.511853 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-12623020-7489-4372-bbe8-1248f501b8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:39.556976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22427e19-73d3-4f84-a246-e759fc4ab7a4 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:39.563735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b7695-c28d-4e65-ac4d-0b9118fdf392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:39.564473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b7695-c28d-4e65-ac4d-0b9118fdf392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:39.565077 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b7695-c28d-4e65-ac4d-0b9118fdf392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:39.565199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b7695-c28d-4e65-ac4d-0b9118fdf392 admin admin] PATH_INFO: `/v3/users/fb8effb6c6e44fbf98a725666a530dce` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:39.566498 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b7695-c28d-4e65-ac4d-0b9118fdf392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb8effb6c6e44fbf98a725666a530dce)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:39.568510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b7695-c28d-4e65-ac4d-0b9118fdf392 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:39.626222 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12623020-7489-4372-bbe8-1248f501b8ad admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:39.633307 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f40469-a0a7-4fe0-9d64-2f19db73744f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:39.634872 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f40469-a0a7-4fe0-9d64-2f19db73744f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:39.635158 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f40469-a0a7-4fe0-9d64-2f19db73744f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:39.635476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f40469-a0a7-4fe0-9d64-2f19db73744f admin admin] PATH_INFO: `/v3/projects/ac3fa915e6d1494a9a5012736272f83e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:39.636789 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f40469-a0a7-4fe0-9d64-2f19db73744f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac3fa915e6d1494a9a5012736272f83e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:39.638906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f40469-a0a7-4fe0-9d64-2f19db73744f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:39.640132 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9a0b7695-c28d-4e65-ac4d-0b9118fdf392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb8effb6c6e44fbf98a725666a530dce'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.658665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9a0b7695-c28d-4e65-ac4d-0b9118fdf392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb8effb6c6e44fbf98a725666a530dce'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.683930 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9a0b7695-c28d-4e65-ac4d-0b9118fdf392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb8effb6c6e44fbf98a725666a530dce'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.710152 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9a0b7695-c28d-4e65-ac4d-0b9118fdf392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb8effb6c6e44fbf98a725666a530dce'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.711747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9a0b7695-c28d-4e65-ac4d-0b9118fdf392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb8effb6c6e44fbf98a725666a530dce'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.724633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1003/2014] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:39 2022] DELETE /identity/v3/users/fb8effb6c6e44fbf98a725666a530dce => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:39.728265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-51f40469-a0a7-4fe0-9d64-2f19db73744f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac3fa915e6d1494a9a5012736272f83e', 'request_id': 'req-51f40469-a0a7-4fe0-9d64-2f19db73744f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.743389 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-51f40469-a0a7-4fe0-9d64-2f19db73744f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac3fa915e6d1494a9a5012736272f83e', 'request_id': 'req-51f40469-a0a7-4fe0-9d64-2f19db73744f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.753705 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-51f40469-a0a7-4fe0-9d64-2f19db73744f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac3fa915e6d1494a9a5012736272f83e', 'request_id': 'req-51f40469-a0a7-4fe0-9d64-2f19db73744f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.755177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:39.775481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-51f40469-a0a7-4fe0-9d64-2f19db73744f admin admin] The token cache is being invalidate because project ac3fa915e6d1494a9a5012736272f83e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:39.776154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-51f40469-a0a7-4fe0-9d64-2f19db73744f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.777119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-51f40469-a0a7-4fe0-9d64-2f19db73744f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:39.781467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1012/2015] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:39 2022] DELETE /identity/v3/projects/ac3fa915e6d1494a9a5012736272f83e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:39.787488 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f1c2b-d55a-461b-b93a-c57b1fc39645 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:39.788001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f1c2b-d55a-461b-b93a-c57b1fc39645 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:39.788541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f1c2b-d55a-461b-b93a-c57b1fc39645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:39.817048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-b94f1c2b-d55a-461b-b93a-c57b1fc39645 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:39.818147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b94f1c2b-d55a-461b-b93a-c57b1fc39645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:39.829975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:39.909171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1004/2016] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:39.910605 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c17f8-3295-4754-83b4-aa1fc9e24d2b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:39.911089 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c17f8-3295-4754-83b4-aa1fc9e24d2b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:39.911089 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c17f8-3295-4754-83b4-aa1fc9e24d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:39.937115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f1c2b-d55a-461b-b93a-c57b1fc39645 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:39.938664 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5c9c17f8-3295-4754-83b4-aa1fc9e24d2b None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:39.940184 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5c9c17f8-3295-4754-83b4-aa1fc9e24d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:39.941926 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b94f1c2b-d55a-461b-b93a-c57b1fc39645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:39.966376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1013/2017] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:39.970072 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f1c2b-d55a-461b-b93a-c57b1fc39645 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:39.970962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c17f8-3295-4754-83b4-aa1fc9e24d2b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:39.989098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fa1ff-4246-42a0-b126-a55b0229970f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:39.989882 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fa1ff-4246-42a0-b126-a55b0229970f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:39.990833 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c17f8-3295-4754-83b4-aa1fc9e24d2b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:39.991232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fa1ff-4246-42a0-b126-a55b0229970f admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:39.991640 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fa1ff-4246-42a0-b126-a55b0229970f admin admin] PATH_INFO: `/v3/projects/a64dc82743ff4b259a52c3985a5fb3e5` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:39.996681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fa1ff-4246-42a0-b126-a55b0229970f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a64dc82743ff4b259a52c3985a5fb3e5)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:39.999700 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fa1ff-4246-42a0-b126-a55b0229970f admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:40.000339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d6450-ce85-4aeb-92e6-1a35d2233844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:40.001139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d6450-ce85-4aeb-92e6-1a35d2233844 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:40.001519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d6450-ce85-4aeb-92e6-1a35d2233844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:40.001832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d6450-ce85-4aeb-92e6-1a35d2233844 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:40.002895 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d6450-ce85-4aeb-92e6-1a35d2233844 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:07:40.004111 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d6450-ce85-4aeb-92e6-1a35d2233844 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:40.006214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d6450-ce85-4aeb-92e6-1a35d2233844 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:40.030000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1014/2018] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:07:40.030608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d6450-ce85-4aeb-92e6-1a35d2233844 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:40.044635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d6450-ce85-4aeb-92e6-1a35d2233844 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:40.050793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b87226-4bed-44f6-9541-015d734bc735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:40.052225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b87226-4bed-44f6-9541-015d734bc735 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:40.052225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b87226-4bed-44f6-9541-015d734bc735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:40.052557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b87226-4bed-44f6-9541-015d734bc735 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:40.053614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b87226-4bed-44f6-9541-015d734bc735 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:40.056460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b87226-4bed-44f6-9541-015d734bc735 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:40.100680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b44fa1ff-4246-42a0-b126-a55b0229970f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a64dc82743ff4b259a52c3985a5fb3e5', 'request_id': 'req-b44fa1ff-4246-42a0-b126-a55b0229970f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:40.106885 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1015/2019] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:07:40 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:40.112247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b87226-4bed-44f6-9541-015d734bc735 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:40.115927 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b44fa1ff-4246-42a0-b126-a55b0229970f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a64dc82743ff4b259a52c3985a5fb3e5', 'request_id': 'req-b44fa1ff-4246-42a0-b126-a55b0229970f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:40.127032 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b44fa1ff-4246-42a0-b126-a55b0229970f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a64dc82743ff4b259a52c3985a5fb3e5', 'request_id': 'req-b44fa1ff-4246-42a0-b126-a55b0229970f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:40.143648 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b44fa1ff-4246-42a0-b126-a55b0229970f admin admin] The token cache is being invalidate because project a64dc82743ff4b259a52c3985a5fb3e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:40.144039 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b44fa1ff-4246-42a0-b126-a55b0229970f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:40.145090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b44fa1ff-4246-42a0-b126-a55b0229970f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:40.147729 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1005/2020] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:39 2022] DELETE /identity/v3/projects/a64dc82743ff4b259a52c3985a5fb3e5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:40.151533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fa1ff-4246-42a0-b126-a55b0229970f admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:40.152933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b44fa1ff-4246-42a0-b126-a55b0229970f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:40.217866 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b87226-4bed-44f6-9541-015d734bc735 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:40.232526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed6dd1-905e-45b6-b4b8-6afb98b0ed97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:40.233166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed6dd1-905e-45b6-b4b8-6afb98b0ed97 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:40.233448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed6dd1-905e-45b6-b4b8-6afb98b0ed97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:40.233720 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed6dd1-905e-45b6-b4b8-6afb98b0ed97 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:40.234406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed6dd1-905e-45b6-b4b8-6afb98b0ed97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:40.236642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed6dd1-905e-45b6-b4b8-6afb98b0ed97 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:40.240038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fa1ff-4246-42a0-b126-a55b0229970f admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:40.245889 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f222431-4edc-466b-bfb0-70f235cebde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:40.246597 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f222431-4edc-466b-bfb0-70f235cebde7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:40.246879 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f222431-4edc-466b-bfb0-70f235cebde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:40.247158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f222431-4edc-466b-bfb0-70f235cebde7 admin admin] PATH_INFO: `/v3/users/351355bb03cb4268b834300db5bb9b1b` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:40.248422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f222431-4edc-466b-bfb0-70f235cebde7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=351355bb03cb4268b834300db5bb9b1b)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:40.250601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f222431-4edc-466b-bfb0-70f235cebde7 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:40.268970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1016/2021] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:07:40 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:40.275355 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed6dd1-905e-45b6-b4b8-6afb98b0ed97 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:40.277798 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c9ed6dd1-905e-45b6-b4b8-6afb98b0ed97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:40.295327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed6dd1-905e-45b6-b4b8-6afb98b0ed97 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:40.301893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ce8b3-f4d0-44e9-82fb-ad706309f27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:40.303045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ce8b3-f4d0-44e9-82fb-ad706309f27b admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:40.303368 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ce8b3-f4d0-44e9-82fb-ad706309f27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:40.303693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ce8b3-f4d0-44e9-82fb-ad706309f27b admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:40.304404 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ce8b3-f4d0-44e9-82fb-ad706309f27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:40.306422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ce8b3-f4d0-44e9-82fb-ad706309f27b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:40.316455 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:40.320982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1017/2022] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:40.325111 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ce8b3-f4d0-44e9-82fb-ad706309f27b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:40.325758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7f222431-4edc-466b-bfb0-70f235cebde7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '351355bb03cb4268b834300db5bb9b1b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:40.327384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5a3ce8b3-f4d0-44e9-82fb-ad706309f27b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:40.338377 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7f222431-4edc-466b-bfb0-70f235cebde7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '351355bb03cb4268b834300db5bb9b1b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:40.345146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ce8b3-f4d0-44e9-82fb-ad706309f27b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:40.350660 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02093f-abe0-4230-bee0-eb598255f530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:40.351830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02093f-abe0-4230-bee0-eb598255f530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:40.352111 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02093f-abe0-4230-bee0-eb598255f530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:40.352383 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02093f-abe0-4230-bee0-eb598255f530 admin admin] PATH_INFO: `/v3/projects/1a8d1a84eabc42cea8f979d027944109` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:40.354422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02093f-abe0-4230-bee0-eb598255f530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a8d1a84eabc42cea8f979d027944109)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:40.355569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7f222431-4edc-466b-bfb0-70f235cebde7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '351355bb03cb4268b834300db5bb9b1b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:40.356331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02093f-abe0-4230-bee0-eb598255f530 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:40.363319 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7f222431-4edc-466b-bfb0-70f235cebde7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '351355bb03cb4268b834300db5bb9b1b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:40.364057 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7f222431-4edc-466b-bfb0-70f235cebde7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '351355bb03cb4268b834300db5bb9b1b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:40.370993 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1006/2023] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:40 2022] DELETE /identity/v3/users/351355bb03cb4268b834300db5bb9b1b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:07:40.371887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f222431-4edc-466b-bfb0-70f235cebde7 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:40.373284 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7f222431-4edc-466b-bfb0-70f235cebde7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:40.418016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9f02093f-abe0-4230-bee0-eb598255f530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a8d1a84eabc42cea8f979d027944109', 'request_id': 'req-9f02093f-abe0-4230-bee0-eb598255f530'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:40.426887 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9f02093f-abe0-4230-bee0-eb598255f530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a8d1a84eabc42cea8f979d027944109', 'request_id': 'req-9f02093f-abe0-4230-bee0-eb598255f530'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:40.433099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9f02093f-abe0-4230-bee0-eb598255f530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a8d1a84eabc42cea8f979d027944109', 'request_id': 'req-9f02093f-abe0-4230-bee0-eb598255f530'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:40.452887 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9f02093f-abe0-4230-bee0-eb598255f530 admin admin] The token cache is being invalidate because project 1a8d1a84eabc42cea8f979d027944109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:40.453427 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9f02093f-abe0-4230-bee0-eb598255f530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:40.454559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9f02093f-abe0-4230-bee0-eb598255f530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:40.456786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1018/2024] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:40 2022] DELETE /identity/v3/projects/1a8d1a84eabc42cea8f979d027944109 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:40.462244 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628b8f0-321b-4554-be4e-d02df31b660b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:40.462824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628b8f0-321b-4554-be4e-d02df31b660b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:40.463294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628b8f0-321b-4554-be4e-d02df31b660b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:40.466296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f222431-4edc-466b-bfb0-70f235cebde7 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:40.495344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-8628b8f0-321b-4554-be4e-d02df31b660b None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:40.497550 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8628b8f0-321b-4554-be4e-d02df31b660b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:40.520961 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:40.547283 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9802fa9-1cfd-4523-a9cb-72f96836277f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:40.547914 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9802fa9-1cfd-4523-a9cb-72f96836277f admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:40.548200 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9802fa9-1cfd-4523-a9cb-72f96836277f admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:40.548474 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9802fa9-1cfd-4523-a9cb-72f96836277f admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:40.549598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9802fa9-1cfd-4523-a9cb-72f96836277f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:40.551278 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9802fa9-1cfd-4523-a9cb-72f96836277f admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:40.567864 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1007/2025] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:40.569932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9802fa9-1cfd-4523-a9cb-72f96836277f admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:40.574631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f9802fa9-1cfd-4523-a9cb-72f96836277f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:40.605872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9802fa9-1cfd-4523-a9cb-72f96836277f admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:40.615293 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f960f-2bb4-476c-be65-1644a67f83db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:40.615986 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f960f-2bb4-476c-be65-1644a67f83db admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:40.616271 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f960f-2bb4-476c-be65-1644a67f83db admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:40.617161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f960f-2bb4-476c-be65-1644a67f83db admin admin] PATH_INFO: `/v3/projects/f85c5e478d5c4efca84cb570537ea15d/users/eef1afc3ece940248494501dd27e3920/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:40.636490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f960f-2bb4-476c-be65-1644a67f83db admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85c5e478d5c4efca84cb570537ea15d, user_id=eef1afc3ece940248494501dd27e3920, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:40.639406 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f960f-2bb4-476c-be65-1644a67f83db admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:40.657959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1019/2026] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:40.670368 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b87e08-037f-4873-b946-fcdf6c07d12b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:40.670681 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b87e08-037f-4873-b946-fcdf6c07d12b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:40.670952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b87e08-037f-4873-b946-fcdf6c07d12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:40.673908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1008/2027] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:40 2022] PUT /identity/v3/projects/f85c5e478d5c4efca84cb570537ea15d/users/eef1afc3ece940248494501dd27e3920/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:40.680303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8a520-1179-4fca-b739-211b59bd2013 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:40.681390 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8a520-1179-4fca-b739-211b59bd2013 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:40.682265 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8a520-1179-4fca-b739-211b59bd2013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:40.729132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-72b87e08-037f-4873-b946-fcdf6c07d12b None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:40.734456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-72b87e08-037f-4873-b946-fcdf6c07d12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:40.736582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-63d8a520-1179-4fca-b739-211b59bd2013 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:40.737990 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-63d8a520-1179-4fca-b739-211b59bd2013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:40.797887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:40.847615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1009/2028] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:40.864002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1020/2029] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:40.868466 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8a520-1179-4fca-b739-211b59bd2013 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:40.876845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b87e08-037f-4873-b946-fcdf6c07d12b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:40.879071 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8a520-1179-4fca-b739-211b59bd2013 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:40.883540 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-72b87e08-037f-4873-b946-fcdf6c07d12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:40.892626 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76945b6a-1103-4d01-ad96-25da55f3332b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:40.893354 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76945b6a-1103-4d01-ad96-25da55f3332b admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:40.893775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76945b6a-1103-4d01-ad96-25da55f3332b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:40.894332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76945b6a-1103-4d01-ad96-25da55f3332b admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:40.895153 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76945b6a-1103-4d01-ad96-25da55f3332b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:07:40.895576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76945b6a-1103-4d01-ad96-25da55f3332b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:40.897920 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76945b6a-1103-4d01-ad96-25da55f3332b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:40.915322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1010/2030] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:07:40.916686 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76945b6a-1103-4d01-ad96-25da55f3332b admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:40.918849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-76945b6a-1103-4d01-ad96-25da55f3332b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:40.941422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76945b6a-1103-4d01-ad96-25da55f3332b admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:40.946692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58f640-3fff-4916-9712-d590afe32a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:40.947514 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58f640-3fff-4916-9712-d590afe32a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:40.947798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58f640-3fff-4916-9712-d590afe32a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:40.949714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58f640-3fff-4916-9712-d590afe32a8e admin admin] PATH_INFO: `/v3/projects/8589ada6a70b4430b07accaf57d9f2d9` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:40.949714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58f640-3fff-4916-9712-d590afe32a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8589ada6a70b4430b07accaf57d9f2d9)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:40.952000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58f640-3fff-4916-9712-d590afe32a8e admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:40.992128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b87e08-037f-4873-b946-fcdf6c07d12b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:41.022343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-de58f640-3fff-4916-9712-d590afe32a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8589ada6a70b4430b07accaf57d9f2d9', 'request_id': 'req-de58f640-3fff-4916-9712-d590afe32a8e'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:41.036677 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-de58f640-3fff-4916-9712-d590afe32a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8589ada6a70b4430b07accaf57d9f2d9', 'request_id': 'req-de58f640-3fff-4916-9712-d590afe32a8e'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:41.046801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-de58f640-3fff-4916-9712-d590afe32a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8589ada6a70b4430b07accaf57d9f2d9', 'request_id': 'req-de58f640-3fff-4916-9712-d590afe32a8e'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:41.062068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-de58f640-3fff-4916-9712-d590afe32a8e admin admin] The token cache is being invalidate because project 8589ada6a70b4430b07accaf57d9f2d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:41.062470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-de58f640-3fff-4916-9712-d590afe32a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:41.063457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-de58f640-3fff-4916-9712-d590afe32a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:41.066534 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1011/2031] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:40 2022] DELETE /identity/v3/projects/8589ada6a70b4430b07accaf57d9f2d9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:41.066534 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58f640-3fff-4916-9712-d590afe32a8e admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:41.066907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-de58f640-3fff-4916-9712-d590afe32a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:41.085385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa3df2-3cbc-4ada-a371-17272de6f60f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:41.086410 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa3df2-3cbc-4ada-a371-17272de6f60f service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:41.086690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa3df2-3cbc-4ada-a371-17272de6f60f service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:41.086973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa3df2-3cbc-4ada-a371-17272de6f60f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:41.089663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-13aa3df2-3cbc-4ada-a371-17272de6f60f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:41.148663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:41.253712 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa3df2-3cbc-4ada-a371-17272de6f60f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:41.255838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa3df2-3cbc-4ada-a371-17272de6f60f service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:41.278481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1021/2032] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:07:41.306790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58f640-3fff-4916-9712-d590afe32a8e admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:41.316366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32f7d9-c269-43ea-b5eb-62bb5f5b0d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:41.317114 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32f7d9-c269-43ea-b5eb-62bb5f5b0d28 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:41.321987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32f7d9-c269-43ea-b5eb-62bb5f5b0d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:41.323113 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32f7d9-c269-43ea-b5eb-62bb5f5b0d28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:41.324457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32f7d9-c269-43ea-b5eb-62bb5f5b0d28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:41.330551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32f7d9-c269-43ea-b5eb-62bb5f5b0d28 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:41.399523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1012/2033] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:07:41 2022] POST /identity/v3/projects => generated 385 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:41.403167 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa3df2-3cbc-4ada-a371-17272de6f60f service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:41.427394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa3df2-3cbc-4ada-a371-17272de6f60f service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:41.519578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91bbd0-8971-47b9-9de0-4c625f3b386f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:41.520364 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91bbd0-8971-47b9-9de0-4c625f3b386f admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:41.520795 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91bbd0-8971-47b9-9de0-4c625f3b386f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:41.521144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91bbd0-8971-47b9-9de0-4c625f3b386f admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:41.521862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91bbd0-8971-47b9-9de0-4c625f3b386f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:41.524472 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91bbd0-8971-47b9-9de0-4c625f3b386f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:41.551413 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1022/2034] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:07:41 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:07:41.555401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32f7d9-c269-43ea-b5eb-62bb5f5b0d28 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:41.568153 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32f7d9-c269-43ea-b5eb-62bb5f5b0d28 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:41.573730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb0b1e-0962-45f9-8f16-7b11cdc634ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:41.574384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb0b1e-0962-45f9-8f16-7b11cdc634ed admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:41.574855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb0b1e-0962-45f9-8f16-7b11cdc634ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:41.575156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb0b1e-0962-45f9-8f16-7b11cdc634ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:41.575838 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb0b1e-0962-45f9-8f16-7b11cdc634ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:41.577557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb0b1e-0962-45f9-8f16-7b11cdc634ed admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:41.593131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1013/2035] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:41.594058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91bbd0-8971-47b9-9de0-4c625f3b386f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:41.605831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91bbd0-8971-47b9-9de0-4c625f3b386f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:41.611850 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fb6dc-b9b6-445a-83e9-612d85184dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:41.612695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fb6dc-b9b6-445a-83e9-612d85184dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:41.613160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fb6dc-b9b6-445a-83e9-612d85184dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:41.613868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fb6dc-b9b6-445a-83e9-612d85184dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:41.614745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fb6dc-b9b6-445a-83e9-612d85184dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:41.616755 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fb6dc-b9b6-445a-83e9-612d85184dd7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:41.634891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb0b1e-0962-45f9-8f16-7b11cdc634ed admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:41.640270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1023/2036] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:07:41.649483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb0b1e-0962-45f9-8f16-7b11cdc634ed admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:41.659333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96acb59-aac5-4a88-a4e9-73a24377a0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:41.660012 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96acb59-aac5-4a88-a4e9-73a24377a0d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:41.660632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96acb59-aac5-4a88-a4e9-73a24377a0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:41.660915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96acb59-aac5-4a88-a4e9-73a24377a0d0 admin admin] PATH_INFO: `/v3/projects/c50e8cf1685f40c791978eae6abfa4d8/users/a3e7c8f141c74a1088acd82bec6a7177/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:41.680862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96acb59-aac5-4a88-a4e9-73a24377a0d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c50e8cf1685f40c791978eae6abfa4d8, user_id=a3e7c8f141c74a1088acd82bec6a7177, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:41.683817 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96acb59-aac5-4a88-a4e9-73a24377a0d0 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:41.692933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:41.701274 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1014/2037] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:41 2022] PUT /identity/v3/projects/c50e8cf1685f40c791978eae6abfa4d8/users/a3e7c8f141c74a1088acd82bec6a7177/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:41.712113 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8969486-385f-4fde-8064-7df5e62be53b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:41.712423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8969486-385f-4fde-8064-7df5e62be53b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:41.713239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8969486-385f-4fde-8064-7df5e62be53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:41.766157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d8969486-385f-4fde-8064-7df5e62be53b None None] MFA Rules not processed for user `a3e7c8f141c74a1088acd82bec6a7177`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:41.767152 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d8969486-385f-4fde-8064-7df5e62be53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:41.844908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1024/2038] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:41.857714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96acb59-aac5-4a88-a4e9-73a24377a0d0 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:41.859895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-c96acb59-aac5-4a88-a4e9-73a24377a0d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:41.937808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96acb59-aac5-4a88-a4e9-73a24377a0d0 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:41.941601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a39cb6-5a61-4542-b98a-3aa9980613e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:41.942247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a39cb6-5a61-4542-b98a-3aa9980613e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:41.942530 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a39cb6-5a61-4542-b98a-3aa9980613e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:41.942802 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a39cb6-5a61-4542-b98a-3aa9980613e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:41.952255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a39cb6-5a61-4542-b98a-3aa9980613e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:41.953697 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a39cb6-5a61-4542-b98a-3aa9980613e9 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:41.965426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1015/2039] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:41.990299 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8969486-385f-4fde-8064-7df5e62be53b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:42.071522 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:42.120616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8969486-385f-4fde-8064-7df5e62be53b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:42.127222 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8086f78-0c86-44cc-a3b7-0d3465fa3141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:42.128080 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8086f78-0c86-44cc-a3b7-0d3465fa3141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:42.128465 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8086f78-0c86-44cc-a3b7-0d3465fa3141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:42.128807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8086f78-0c86-44cc-a3b7-0d3465fa3141 admin admin] PATH_INFO: `/v3/users/a3e7c8f141c74a1088acd82bec6a7177` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:42.130183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8086f78-0c86-44cc-a3b7-0d3465fa3141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3e7c8f141c74a1088acd82bec6a7177)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:42.132613 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8086f78-0c86-44cc-a3b7-0d3465fa3141 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:42.197882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f8086f78-0c86-44cc-a3b7-0d3465fa3141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3e7c8f141c74a1088acd82bec6a7177'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:42.197882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f8086f78-0c86-44cc-a3b7-0d3465fa3141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3e7c8f141c74a1088acd82bec6a7177'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:42.203636 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f8086f78-0c86-44cc-a3b7-0d3465fa3141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3e7c8f141c74a1088acd82bec6a7177'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:42.213221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f8086f78-0c86-44cc-a3b7-0d3465fa3141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3e7c8f141c74a1088acd82bec6a7177'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:42.230696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f8086f78-0c86-44cc-a3b7-0d3465fa3141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3e7c8f141c74a1088acd82bec6a7177'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:42.240718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1025/2040] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:07:41 2022] DELETE /identity/v3/users/a3e7c8f141c74a1088acd82bec6a7177 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:42.242182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8615a6-2eb9-43c9-9500-c7f5b86b6224 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:42.242673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8615a6-2eb9-43c9-9500-c7f5b86b6224 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:42.242781 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8615a6-2eb9-43c9-9500-c7f5b86b6224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:42.266218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-7d8615a6-2eb9-43c9-9500-c7f5b86b6224 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:42.268192 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7d8615a6-2eb9-43c9-9500-c7f5b86b6224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:42.342988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1016/2041] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:07:42.353695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8086f78-0c86-44cc-a3b7-0d3465fa3141 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:42.355677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f8086f78-0c86-44cc-a3b7-0d3465fa3141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:42.451058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8086f78-0c86-44cc-a3b7-0d3465fa3141 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:42.456687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a3c4f-d16b-4618-806a-d31ed52ee19e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:42.458453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a3c4f-d16b-4618-806a-d31ed52ee19e service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:42.458904 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a3c4f-d16b-4618-806a-d31ed52ee19e service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:42.459323 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a3c4f-d16b-4618-806a-d31ed52ee19e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:42.477080 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a3c4f-d16b-4618-806a-d31ed52ee19e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:42.478817 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a3c4f-d16b-4618-806a-d31ed52ee19e service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:42.491383 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1026/2042] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:42.744474 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33c5a0-26ae-429b-a97f-807dc70831e7 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:42.745296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33c5a0-26ae-429b-a97f-807dc70831e7 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:42.746054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33c5a0-26ae-429b-a97f-807dc70831e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:42.815258 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-6c33c5a0-26ae-429b-a97f-807dc70831e7 None None] MFA Rules not processed for user `eef1afc3ece940248494501dd27e3920`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:42.819862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6c33c5a0-26ae-429b-a97f-807dc70831e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:42.956417 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1017/2043] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:42.969148 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a3c4f-d16b-4618-806a-d31ed52ee19e service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:42.971122 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c25a3c4f-d16b-4618-806a-d31ed52ee19e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:43.021473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33c5a0-26ae-429b-a97f-807dc70831e7 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:43.023701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6c33c5a0-26ae-429b-a97f-807dc70831e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:43.042465 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33c5a0-26ae-429b-a97f-807dc70831e7 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:43.048857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c936826-1dff-4be4-a427-67213855d284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:43.049744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c936826-1dff-4be4-a427-67213855d284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:43.050090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c936826-1dff-4be4-a427-67213855d284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:43.050487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c936826-1dff-4be4-a427-67213855d284 admin admin] PATH_INFO: `/v3/projects/c50e8cf1685f40c791978eae6abfa4d8` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:43.051868 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c936826-1dff-4be4-a427-67213855d284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c50e8cf1685f40c791978eae6abfa4d8)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:43.054179 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c936826-1dff-4be4-a427-67213855d284 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:43.066528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a3c4f-d16b-4618-806a-d31ed52ee19e service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:43.071527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d12d7-758c-49bc-8db2-54ce57a49dd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:43.072310 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d12d7-758c-49bc-8db2-54ce57a49dd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:43.072880 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d12d7-758c-49bc-8db2-54ce57a49dd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:43.072960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d12d7-758c-49bc-8db2-54ce57a49dd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:43.079994 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:43.084756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d12d7-758c-49bc-8db2-54ce57a49dd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:43.086719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d12d7-758c-49bc-8db2-54ce57a49dd5 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:43.099445 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:43.100613 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1027/2044] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:43.123823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-8c936826-1dff-4be4-a427-67213855d284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c50e8cf1685f40c791978eae6abfa4d8', 'request_id': 'req-8c936826-1dff-4be4-a427-67213855d284'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:43.134875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-8c936826-1dff-4be4-a427-67213855d284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c50e8cf1685f40c791978eae6abfa4d8', 'request_id': 'req-8c936826-1dff-4be4-a427-67213855d284'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:43.142634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-8c936826-1dff-4be4-a427-67213855d284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c50e8cf1685f40c791978eae6abfa4d8', 'request_id': 'req-8c936826-1dff-4be4-a427-67213855d284'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:43.156688 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-8c936826-1dff-4be4-a427-67213855d284 admin admin] The token cache is being invalidate because project c50e8cf1685f40c791978eae6abfa4d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:07:43.157133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-8c936826-1dff-4be4-a427-67213855d284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:43.157674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-8c936826-1dff-4be4-a427-67213855d284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:07:43.159813 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1018/2045] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:07:43 2022] DELETE /identity/v3/projects/c50e8cf1685f40c791978eae6abfa4d8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:07:43.164549 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cd8cb-c16c-4f6b-9769-90ccaba8582d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:43.164832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cd8cb-c16c-4f6b-9769-90ccaba8582d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:43.165104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cd8cb-c16c-4f6b-9769-90ccaba8582d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:43.190899 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f18cd8cb-c16c-4f6b-9769-90ccaba8582d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:43.197061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f18cd8cb-c16c-4f6b-9769-90ccaba8582d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:43.279119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1028/2046] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:43.287434 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdae85-3de5-4b3d-9fc6-b4ca6b44f0b6 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:43.287930 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdae85-3de5-4b3d-9fc6-b4ca6b44f0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:43.288028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdae85-3de5-4b3d-9fc6-b4ca6b44f0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:43.310001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-91cdae85-3de5-4b3d-9fc6-b4ca6b44f0b6 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:43.310472 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-91cdae85-3de5-4b3d-9fc6-b4ca6b44f0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:43.321445 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1019/2047] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:43.325517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cd8cb-c16c-4f6b-9769-90ccaba8582d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:43.336360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cd8cb-c16c-4f6b-9769-90ccaba8582d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:43.342990 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da27bd-553f-4b7b-abba-339203ac33fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:43.344086 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da27bd-553f-4b7b-abba-339203ac33fc admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:43.344370 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da27bd-553f-4b7b-abba-339203ac33fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:43.344998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da27bd-553f-4b7b-abba-339203ac33fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:43.346285 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da27bd-553f-4b7b-abba-339203ac33fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:07:43.346637 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da27bd-553f-4b7b-abba-339203ac33fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:43.349253 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da27bd-553f-4b7b-abba-339203ac33fc admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:43.365350 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1029/2048] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:07:43.369605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdae85-3de5-4b3d-9fc6-b4ca6b44f0b6 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:43.379765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdae85-3de5-4b3d-9fc6-b4ca6b44f0b6 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:43.386048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddcbc2-18c7-4b03-925f-c92fb1b2e4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:43.386701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddcbc2-18c7-4b03-925f-c92fb1b2e4ca admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:43.386980 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddcbc2-18c7-4b03-925f-c92fb1b2e4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:43.388328 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddcbc2-18c7-4b03-925f-c92fb1b2e4ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:43.388328 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddcbc2-18c7-4b03-925f-c92fb1b2e4ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:43.390805 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddcbc2-18c7-4b03-925f-c92fb1b2e4ca admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:43.445631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1020/2049] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:07:43 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:07:43.447189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da27bd-553f-4b7b-abba-339203ac33fc admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:43.459091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da27bd-553f-4b7b-abba-339203ac33fc admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:43.536926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37a9ed-6dc9-4630-a605-0a58327f41b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:43.537553 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37a9ed-6dc9-4630-a605-0a58327f41b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:43.537836 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37a9ed-6dc9-4630-a605-0a58327f41b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:43.538110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37a9ed-6dc9-4630-a605-0a58327f41b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:43.538827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37a9ed-6dc9-4630-a605-0a58327f41b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:43.540801 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37a9ed-6dc9-4630-a605-0a58327f41b9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:43.566551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1030/2050] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:07:43 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:07:43.568051 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddcbc2-18c7-4b03-925f-c92fb1b2e4ca admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:43.580239 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddcbc2-18c7-4b03-925f-c92fb1b2e4ca admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:43.586212 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18345f84-8840-489d-b7f0-dc6d2f3035b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:43.586212 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18345f84-8840-489d-b7f0-dc6d2f3035b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:43.586212 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18345f84-8840-489d-b7f0-dc6d2f3035b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:43.586212 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18345f84-8840-489d-b7f0-dc6d2f3035b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:43.586856 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18345f84-8840-489d-b7f0-dc6d2f3035b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:43.588499 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18345f84-8840-489d-b7f0-dc6d2f3035b2 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:43.602154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1021/2051] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:43.604028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37a9ed-6dc9-4630-a605-0a58327f41b9 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:43.614942 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37a9ed-6dc9-4630-a605-0a58327f41b9 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:43.620627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce027ada-dbf0-4a54-84ef-dcdf175198c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:43.621257 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce027ada-dbf0-4a54-84ef-dcdf175198c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:43.621664 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce027ada-dbf0-4a54-84ef-dcdf175198c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:43.621935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce027ada-dbf0-4a54-84ef-dcdf175198c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:43.622614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce027ada-dbf0-4a54-84ef-dcdf175198c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:43.624241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce027ada-dbf0-4a54-84ef-dcdf175198c3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:43.636505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1031/2052] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:43.640611 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18345f84-8840-489d-b7f0-dc6d2f3035b2 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:43.651861 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18345f84-8840-489d-b7f0-dc6d2f3035b2 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:43.657207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e1534-b898-43cb-ac90-11cc4899b3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:43.657951 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e1534-b898-43cb-ac90-11cc4899b3c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:43.658288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e1534-b898-43cb-ac90-11cc4899b3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:43.658568 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e1534-b898-43cb-ac90-11cc4899b3c5 admin admin] PATH_INFO: `/v3/projects/9ffe74cee98b482ab7cdda8a452c01e2/users/d6a3627d555b4e12be83d5b712ad3bee/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:43.675188 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e1534-b898-43cb-ac90-11cc4899b3c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ffe74cee98b482ab7cdda8a452c01e2, user_id=d6a3627d555b4e12be83d5b712ad3bee, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:43.677898 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e1534-b898-43cb-ac90-11cc4899b3c5 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:43.698543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1022/2053] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:07:43 2022] PUT /identity/v3/projects/9ffe74cee98b482ab7cdda8a452c01e2/users/d6a3627d555b4e12be83d5b712ad3bee/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:07:43.713373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c1a02-8312-4816-b200-57d4d01c211a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:43.713373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c1a02-8312-4816-b200-57d4d01c211a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:43.713373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c1a02-8312-4816-b200-57d4d01c211a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:43.754973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-bc3c1a02-8312-4816-b200-57d4d01c211a None None] MFA Rules not processed for user `d6a3627d555b4e12be83d5b712ad3bee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:43.756400 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bc3c1a02-8312-4816-b200-57d4d01c211a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:43.821414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:43.853518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1032/2054] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:07:43.868967 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e1534-b898-43cb-ac90-11cc4899b3c5 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:43.871086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-8d7e1534-b898-43cb-ac90-11cc4899b3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:43.937531 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e1534-b898-43cb-ac90-11cc4899b3c5 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:43.941547 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0749ec9-412c-49f7-afdf-0fe2162224e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:43.942187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0749ec9-412c-49f7-afdf-0fe2162224e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:43.942469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0749ec9-412c-49f7-afdf-0fe2162224e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:43.942738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0749ec9-412c-49f7-afdf-0fe2162224e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:43.952159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0749ec9-412c-49f7-afdf-0fe2162224e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:43.953600 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0749ec9-412c-49f7-afdf-0fe2162224e0 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:43.963466 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1023/2055] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:45.609963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee24a3-cbe0-4e3f-ab2f-b1bf1055e998 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:45.610584 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee24a3-cbe0-4e3f-ab2f-b1bf1055e998 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:45.610584 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee24a3-cbe0-4e3f-ab2f-b1bf1055e998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:45.629600 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-6aee24a3-cbe0-4e3f-ab2f-b1bf1055e998 None None] MFA Rules not processed for user `eef1afc3ece940248494501dd27e3920`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:45.630838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6aee24a3-cbe0-4e3f-ab2f-b1bf1055e998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:45.689481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1033/2056] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:07:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:53.958950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:55.681990 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=30 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:04.156433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470603a-01ec-4d03-bc3a-ff109378c4d0 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:04.157524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470603a-01ec-4d03-bc3a-ff109378c4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:04.158046 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470603a-01ec-4d03-bc3a-ff109378c4d0 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:04.160563 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1024/2057] 10.222.0.62 () {54 vars in 921 bytes} [Fri Sep 9 12:08:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:08:04.171030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516f77c-9b9b-4a72-85d4-6a80bbecdad6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:04.171861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516f77c-9b9b-4a72-85d4-6a80bbecdad6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:04.172286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516f77c-9b9b-4a72-85d4-6a80bbecdad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:04.199670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7516f77c-9b9b-4a72-85d4-6a80bbecdad6 None None] MFA Rules not processed for user `c58bae3ef7784671ae247f4ad4199c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:04.201203 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7516f77c-9b9b-4a72-85d4-6a80bbecdad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:04.275823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470603a-01ec-4d03-bc3a-ff109378c4d0 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:04.278602 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2470603a-01ec-4d03-bc3a-ff109378c4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:04.300118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1034/2058] 10.222.0.62 () {58 vars in 1018 bytes} [Fri Sep 9 12:08:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:04.315805 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970ff28-1c24-4d54-a554-cb8f6fa5bcd6 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:04.316140 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970ff28-1c24-4d54-a554-cb8f6fa5bcd6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:04.316417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970ff28-1c24-4d54-a554-cb8f6fa5bcd6 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:04.318152 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1035/2059] 10.222.0.62 () {54 vars in 961 bytes} [Fri Sep 9 12:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:08:04.324250 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a02b0-dfaa-465f-91d5-83d65763c546 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:04.324760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a02b0-dfaa-465f-91d5-83d65763c546 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:04.325228 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a02b0-dfaa-465f-91d5-83d65763c546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:04.356188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-498a02b0-dfaa-465f-91d5-83d65763c546 None None] MFA Rules not processed for user `c58bae3ef7784671ae247f4ad4199c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:04.358285 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-498a02b0-dfaa-465f-91d5-83d65763c546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:04.372932 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1036/2060] 10.222.0.62 () {58 vars in 1058 bytes} [Fri Sep 9 12:08:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:04.382281 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a02b0-dfaa-465f-91d5-83d65763c546 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:04.393170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a02b0-dfaa-465f-91d5-83d65763c546 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:04.394725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470603a-01ec-4d03-bc3a-ff109378c4d0 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:04.399259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7135993-eff5-4791-96bb-7058f6a258c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:04.399855 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7135993-eff5-4791-96bb-7058f6a258c9 service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:04.400127 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7135993-eff5-4791-96bb-7058f6a258c9 service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:04.400393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7135993-eff5-4791-96bb-7058f6a258c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:04.401519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfdfdf-7fa0-4bf0-b4aa-8340d2410be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:04.402188 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfdfdf-7fa0-4bf0-b4aa-8340d2410be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:04.402473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfdfdf-7fa0-4bf0-b4aa-8340d2410be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:04.402813 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfdfdf-7fa0-4bf0-b4aa-8340d2410be9 admin admin] PATH_INFO: `/v3/users/326c59915e3d4b7289472d1400730a07` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:04.406066 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfdfdf-7fa0-4bf0-b4aa-8340d2410be9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=326c59915e3d4b7289472d1400730a07)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:04.408078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfdfdf-7fa0-4bf0-b4aa-8340d2410be9 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:04.415762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7135993-eff5-4791-96bb-7058f6a258c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:04.417321 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7135993-eff5-4791-96bb-7058f6a258c9 service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:04.428868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1037/2061] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:08:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:08:04.465094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-64cfdfdf-7fa0-4bf0-b4aa-8340d2410be9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326c59915e3d4b7289472d1400730a07'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:04.474034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-64cfdfdf-7fa0-4bf0-b4aa-8340d2410be9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326c59915e3d4b7289472d1400730a07'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:04.486738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-64cfdfdf-7fa0-4bf0-b4aa-8340d2410be9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326c59915e3d4b7289472d1400730a07'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:04.493637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-64cfdfdf-7fa0-4bf0-b4aa-8340d2410be9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326c59915e3d4b7289472d1400730a07'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:04.494454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-64cfdfdf-7fa0-4bf0-b4aa-8340d2410be9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326c59915e3d4b7289472d1400730a07'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:04.501559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1025/2062] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:08:04 2022] DELETE /identity/v3/users/326c59915e3d4b7289472d1400730a07 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:08:04.696084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7135993-eff5-4791-96bb-7058f6a258c9 service placement] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:04.707796 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7135993-eff5-4791-96bb-7058f6a258c9 service placement] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:04.766272 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3550c-22c8-4265-acc0-86ddacc9b15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:04.766907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3550c-22c8-4265-acc0-86ddacc9b15e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:04.767188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3550c-22c8-4265-acc0-86ddacc9b15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:04.767460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3550c-22c8-4265-acc0-86ddacc9b15e admin admin] PATH_INFO: `/v3/projects/19d3583d753d4efab0bcd6769574420b` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:04.768586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3550c-22c8-4265-acc0-86ddacc9b15e admin admin] RBAC: Authorizing `identity:delete_project(project_id=19d3583d753d4efab0bcd6769574420b)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:04.770528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3550c-22c8-4265-acc0-86ddacc9b15e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:04.817876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-73f3550c-22c8-4265-acc0-86ddacc9b15e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19d3583d753d4efab0bcd6769574420b', 'request_id': 'req-73f3550c-22c8-4265-acc0-86ddacc9b15e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:04.826590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-73f3550c-22c8-4265-acc0-86ddacc9b15e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19d3583d753d4efab0bcd6769574420b', 'request_id': 'req-73f3550c-22c8-4265-acc0-86ddacc9b15e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:04.835184 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-73f3550c-22c8-4265-acc0-86ddacc9b15e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19d3583d753d4efab0bcd6769574420b', 'request_id': 'req-73f3550c-22c8-4265-acc0-86ddacc9b15e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:04.850433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-73f3550c-22c8-4265-acc0-86ddacc9b15e admin admin] The token cache is being invalidate because project 19d3583d753d4efab0bcd6769574420b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:08:04.850720 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-73f3550c-22c8-4265-acc0-86ddacc9b15e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:04.851483 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-73f3550c-22c8-4265-acc0-86ddacc9b15e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:04.853003 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1038/2063] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:08:04 2022] DELETE /identity/v3/projects/19d3583d753d4efab0bcd6769574420b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:04.857989 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038f3e8-19c5-430f-9975-ac90d2a71d5f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:04.858300 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038f3e8-19c5-430f-9975-ac90d2a71d5f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:04.858610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038f3e8-19c5-430f-9975-ac90d2a71d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:04.880424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d038f3e8-19c5-430f-9975-ac90d2a71d5f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:04.882339 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d038f3e8-19c5-430f-9975-ac90d2a71d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:04.944570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1026/2064] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:04.952471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f96677-6db4-40d1-9b67-05a1177cc39c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:04.952784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f96677-6db4-40d1-9b67-05a1177cc39c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:04.953059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f96677-6db4-40d1-9b67-05a1177cc39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:04.967340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:04.972811 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-32f96677-6db4-40d1-9b67-05a1177cc39c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:04.974814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-32f96677-6db4-40d1-9b67-05a1177cc39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:04.984014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1039/2065] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:04.988453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038f3e8-19c5-430f-9975-ac90d2a71d5f None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:04.999070 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038f3e8-19c5-430f-9975-ac90d2a71d5f None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:05.003863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed84ed3-7319-48b9-9167-e46fe987a584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:05.004482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed84ed3-7319-48b9-9167-e46fe987a584 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:05.004761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed84ed3-7319-48b9-9167-e46fe987a584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:05.005034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed84ed3-7319-48b9-9167-e46fe987a584 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:05.005678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed84ed3-7319-48b9-9167-e46fe987a584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:08:05.005979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed84ed3-7319-48b9-9167-e46fe987a584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:05.007682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed84ed3-7319-48b9-9167-e46fe987a584 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:05.019670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1027/2066] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:08:05.023951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f96677-6db4-40d1-9b67-05a1177cc39c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:05.033191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f96677-6db4-40d1-9b67-05a1177cc39c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:05.038538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24f5ea-3954-4927-bede-ebaffe23a7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:05.039133 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24f5ea-3954-4927-bede-ebaffe23a7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:05.039408 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24f5ea-3954-4927-bede-ebaffe23a7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:05.039682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24f5ea-3954-4927-bede-ebaffe23a7f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:05.040320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24f5ea-3954-4927-bede-ebaffe23a7f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:05.042296 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24f5ea-3954-4927-bede-ebaffe23a7f8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:05.070794 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1040/2067] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:08:05 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:05.075569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed84ed3-7319-48b9-9167-e46fe987a584 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:05.086671 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed84ed3-7319-48b9-9167-e46fe987a584 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:05.090204 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:05.146692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be632c1c-d279-4632-a8f7-6905308d2ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:05.147409 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be632c1c-d279-4632-a8f7-6905308d2ffd admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:05.147991 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be632c1c-d279-4632-a8f7-6905308d2ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:05.147991 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be632c1c-d279-4632-a8f7-6905308d2ffd admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:05.148477 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be632c1c-d279-4632-a8f7-6905308d2ffd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:05.150443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be632c1c-d279-4632-a8f7-6905308d2ffd admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:05.180381 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1028/2068] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:08:05 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:05.184560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24f5ea-3954-4927-bede-ebaffe23a7f8 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:05.195556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24f5ea-3954-4927-bede-ebaffe23a7f8 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:05.201103 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84deb2ca-f583-4bae-bd3f-34563f47d894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:05.201714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84deb2ca-f583-4bae-bd3f-34563f47d894 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:05.201999 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84deb2ca-f583-4bae-bd3f-34563f47d894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:05.202308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84deb2ca-f583-4bae-bd3f-34563f47d894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:05.202953 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84deb2ca-f583-4bae-bd3f-34563f47d894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:05.204541 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84deb2ca-f583-4bae-bd3f-34563f47d894 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:05.214926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1041/2069] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:08:05.219420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be632c1c-d279-4632-a8f7-6905308d2ffd admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:05.230283 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be632c1c-d279-4632-a8f7-6905308d2ffd admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:05.234973 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038007d2-5d53-4fb1-8eea-b2ce338d5cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:05.235569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038007d2-5d53-4fb1-8eea-b2ce338d5cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:05.235842 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038007d2-5d53-4fb1-8eea-b2ce338d5cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:05.236111 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038007d2-5d53-4fb1-8eea-b2ce338d5cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:05.236754 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038007d2-5d53-4fb1-8eea-b2ce338d5cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:05.238401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038007d2-5d53-4fb1-8eea-b2ce338d5cb4 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:05.249909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1029/2070] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:08:05.254269 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84deb2ca-f583-4bae-bd3f-34563f47d894 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:05.264394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84deb2ca-f583-4bae-bd3f-34563f47d894 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:05.270240 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9ffd3-7ff9-4b16-b4b5-6130d77e4ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:05.270883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9ffd3-7ff9-4b16-b4b5-6130d77e4ab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:05.271166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9ffd3-7ff9-4b16-b4b5-6130d77e4ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:05.271431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9ffd3-7ff9-4b16-b4b5-6130d77e4ab4 admin admin] PATH_INFO: `/v3/projects/135fcd62ab6a49b8828c360cf95f41e3/users/af3de82e9ce74d49a72cc5d14e5e3d67/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:05.286341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9ffd3-7ff9-4b16-b4b5-6130d77e4ab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=135fcd62ab6a49b8828c360cf95f41e3, user_id=af3de82e9ce74d49a72cc5d14e5e3d67, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:05.289604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9ffd3-7ff9-4b16-b4b5-6130d77e4ab4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:05.302991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1042/2071] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:08:05 2022] PUT /identity/v3/projects/135fcd62ab6a49b8828c360cf95f41e3/users/af3de82e9ce74d49a72cc5d14e5e3d67/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:05.314505 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637943b-da9e-46b3-af14-205fa12e7ad7 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:05.314789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637943b-da9e-46b3-af14-205fa12e7ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:05.315065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637943b-da9e-46b3-af14-205fa12e7ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:05.343281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-4637943b-da9e-46b3-af14-205fa12e7ad7 None None] MFA Rules not processed for user `af3de82e9ce74d49a72cc5d14e5e3d67`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:05.344533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-4637943b-da9e-46b3-af14-205fa12e7ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:05.411059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1030/2072] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:05.415381 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9ffd3-7ff9-4b16-b4b5-6130d77e4ab4 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:05.477202 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9ffd3-7ff9-4b16-b4b5-6130d77e4ab4 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:05.481838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef82df-ba0b-4ee1-9431-b437d133fa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:05.482473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef82df-ba0b-4ee1-9431-b437d133fa85 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:05.482749 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef82df-ba0b-4ee1-9431-b437d133fa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:05.483016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef82df-ba0b-4ee1-9431-b437d133fa85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:05.483652 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef82df-ba0b-4ee1-9431-b437d133fa85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:05.485542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef82df-ba0b-4ee1-9431-b437d133fa85 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:05.521396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1043/2073] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:08:05 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:05.522632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637943b-da9e-46b3-af14-205fa12e7ad7 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:05.533479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637943b-da9e-46b3-af14-205fa12e7ad7 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:05.590018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e80597-2e24-46cf-8cdb-c4818b0972aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:05.590657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e80597-2e24-46cf-8cdb-c4818b0972aa admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:05.590933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e80597-2e24-46cf-8cdb-c4818b0972aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:05.591201 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e80597-2e24-46cf-8cdb-c4818b0972aa admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:05.591831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e80597-2e24-46cf-8cdb-c4818b0972aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:05.593793 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e80597-2e24-46cf-8cdb-c4818b0972aa admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:05.620640 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1031/2074] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:08:05 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:05.622068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef82df-ba0b-4ee1-9431-b437d133fa85 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:05.635865 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef82df-ba0b-4ee1-9431-b437d133fa85 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:05.640940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ac458-d0fc-4ab2-86fe-4611b02836b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:05.641559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ac458-d0fc-4ab2-86fe-4611b02836b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:05.641838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ac458-d0fc-4ab2-86fe-4611b02836b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:05.642110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ac458-d0fc-4ab2-86fe-4611b02836b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:05.643566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ac458-d0fc-4ab2-86fe-4611b02836b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:05.645222 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ac458-d0fc-4ab2-86fe-4611b02836b7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:05.662173 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1044/2075] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:08:05.664639 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e80597-2e24-46cf-8cdb-c4818b0972aa admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:05.671365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:05.675841 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e80597-2e24-46cf-8cdb-c4818b0972aa admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:05.682784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08dd88-107c-4a04-90c2-57ea8b499c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:05.683448 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08dd88-107c-4a04-90c2-57ea8b499c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:05.683753 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08dd88-107c-4a04-90c2-57ea8b499c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:05.684027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08dd88-107c-4a04-90c2-57ea8b499c19 admin admin] PATH_INFO: `/v3/projects/fc4b0757c98449039cf2656c0c5d36c8/users/f745c28a6b3243518953bc8987e548ad/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:05.701042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08dd88-107c-4a04-90c2-57ea8b499c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4b0757c98449039cf2656c0c5d36c8, user_id=f745c28a6b3243518953bc8987e548ad, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:05.703768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08dd88-107c-4a04-90c2-57ea8b499c19 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:05.717720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1032/2076] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:08:05 2022] PUT /identity/v3/projects/fc4b0757c98449039cf2656c0c5d36c8/users/f745c28a6b3243518953bc8987e548ad/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:05.722001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ac458-d0fc-4ab2-86fe-4611b02836b7 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:05.732939 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ac458-d0fc-4ab2-86fe-4611b02836b7 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:05.792031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820888f0-5a36-4153-9a43-97ee922ef1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:05.792643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820888f0-5a36-4153-9a43-97ee922ef1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:05.792920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820888f0-5a36-4153-9a43-97ee922ef1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:05.793202 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820888f0-5a36-4153-9a43-97ee922ef1d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:05.793845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820888f0-5a36-4153-9a43-97ee922ef1d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:05.795501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820888f0-5a36-4153-9a43-97ee922ef1d7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:05.805780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1045/2077] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:08:05.810238 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08dd88-107c-4a04-90c2-57ea8b499c19 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:05.820765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08dd88-107c-4a04-90c2-57ea8b499c19 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:05.825326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb9cb6-22d7-4be9-82d3-8c5326bafbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:05.826003 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb9cb6-22d7-4be9-82d3-8c5326bafbaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:05.826414 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb9cb6-22d7-4be9-82d3-8c5326bafbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:05.826818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb9cb6-22d7-4be9-82d3-8c5326bafbaf admin admin] PATH_INFO: `/v3/projects/fc4b0757c98449039cf2656c0c5d36c8/users/f745c28a6b3243518953bc8987e548ad/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:05.829165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb9cb6-22d7-4be9-82d3-8c5326bafbaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4b0757c98449039cf2656c0c5d36c8, user_id=f745c28a6b3243518953bc8987e548ad, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:05.832201 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb9cb6-22d7-4be9-82d3-8c5326bafbaf admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:05.842715 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1033/2078] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:08:05 2022] PUT /identity/v3/projects/fc4b0757c98449039cf2656c0c5d36c8/users/f745c28a6b3243518953bc8987e548ad/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:05.853583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e4d80-9e66-4025-827f-15efa725d69d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:05.853797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e4d80-9e66-4025-827f-15efa725d69d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:05.854071 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e4d80-9e66-4025-827f-15efa725d69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:05.879377 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:05.883081 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e53e4d80-9e66-4025-827f-15efa725d69d None None] MFA Rules not processed for user `f745c28a6b3243518953bc8987e548ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:05.885089 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e53e4d80-9e66-4025-827f-15efa725d69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:05.954062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1046/2079] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:05.965501 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb9cb6-22d7-4be9-82d3-8c5326bafbaf admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:05.967244 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-03fb9cb6-22d7-4be9-82d3-8c5326bafbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:06.027475 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb9cb6-22d7-4be9-82d3-8c5326bafbaf admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:06.033401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce8632-f112-4db1-90b9-ea8f37c7a793 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:06.034096 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce8632-f112-4db1-90b9-ea8f37c7a793 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:06.034527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce8632-f112-4db1-90b9-ea8f37c7a793 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:06.034871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce8632-f112-4db1-90b9-ea8f37c7a793 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:06.091574 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce8632-f112-4db1-90b9-ea8f37c7a793 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:06.093002 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce8632-f112-4db1-90b9-ea8f37c7a793 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:06.101772 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1034/2080] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:08:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:08:14.898096 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e4d80-9e66-4025-827f-15efa725d69d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:14.907509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e4d80-9e66-4025-827f-15efa725d69d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:14.911108 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b65db-6d6c-4cda-9c60-c443375f4252 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:14.911701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b65db-6d6c-4cda-9c60-c443375f4252 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:14.911976 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b65db-6d6c-4cda-9c60-c443375f4252 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:14.912247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b65db-6d6c-4cda-9c60-c443375f4252 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:14.921313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b65db-6d6c-4cda-9c60-c443375f4252 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:14.922754 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b65db-6d6c-4cda-9c60-c443375f4252 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:14.932274 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1047/2081] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:08:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:08:16.097796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=61,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:23.004398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce8632-f112-4db1-90b9-ea8f37c7a793 service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:23.008176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-84ce8632-f112-4db1-90b9-ea8f37c7a793 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:23.105732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce8632-f112-4db1-90b9-ea8f37c7a793 service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:23.110936 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d9c5-dc0f-4c0e-8d7f-3cae56c2baf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:23.111587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d9c5-dc0f-4c0e-8d7f-3cae56c2baf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:23.111886 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d9c5-dc0f-4c0e-8d7f-3cae56c2baf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:23.112164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d9c5-dc0f-4c0e-8d7f-3cae56c2baf4 admin admin] PATH_INFO: `/v3/users/d6a3627d555b4e12be83d5b712ad3bee` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:23.121752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201d9c5-dc0f-4c0e-8d7f-3cae56c2baf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6a3627d555b4e12be83d5b712ad3bee)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:23.128128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201d9c5-dc0f-4c0e-8d7f-3cae56c2baf4 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:23.198716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2201d9c5-dc0f-4c0e-8d7f-3cae56c2baf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6a3627d555b4e12be83d5b712ad3bee'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:23.208492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2201d9c5-dc0f-4c0e-8d7f-3cae56c2baf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6a3627d555b4e12be83d5b712ad3bee'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:23.227090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2201d9c5-dc0f-4c0e-8d7f-3cae56c2baf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6a3627d555b4e12be83d5b712ad3bee'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:23.236098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2201d9c5-dc0f-4c0e-8d7f-3cae56c2baf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6a3627d555b4e12be83d5b712ad3bee'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:23.237031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2201d9c5-dc0f-4c0e-8d7f-3cae56c2baf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6a3627d555b4e12be83d5b712ad3bee'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:23.244157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1035/2082] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:08:23 2022] DELETE /identity/v3/users/d6a3627d555b4e12be83d5b712ad3bee => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:23.248902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b72e1-0480-42d9-acaf-73803d47ab0a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:23.249392 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b72e1-0480-42d9-acaf-73803d47ab0a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:23.249492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b72e1-0480-42d9-acaf-73803d47ab0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:23.269813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-247b72e1-0480-42d9-acaf-73803d47ab0a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:23.271081 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-247b72e1-0480-42d9-acaf-73803d47ab0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:23.333039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1048/2083] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:08:23.343354 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d9c5-dc0f-4c0e-8d7f-3cae56c2baf4 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:23.344839 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2201d9c5-dc0f-4c0e-8d7f-3cae56c2baf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:23.403387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d9c5-dc0f-4c0e-8d7f-3cae56c2baf4 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:23.407019 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8d80b-fe45-4944-ad0e-f50cad72e5bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:23.407728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8d80b-fe45-4944-ad0e-f50cad72e5bc service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:23.408004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8d80b-fe45-4944-ad0e-f50cad72e5bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:23.408272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8d80b-fe45-4944-ad0e-f50cad72e5bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:23.418756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8d80b-fe45-4944-ad0e-f50cad72e5bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:23.420193 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8d80b-fe45-4944-ad0e-f50cad72e5bc service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:23.429609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1036/2084] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:08:23.767472 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b72e1-0480-42d9-acaf-73803d47ab0a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:23.768913 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-247b72e1-0480-42d9-acaf-73803d47ab0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:23.782027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b72e1-0480-42d9-acaf-73803d47ab0a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:23.786728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875eed7-e877-43e1-90d8-2dfced75e5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:23.787365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875eed7-e877-43e1-90d8-2dfced75e5cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:23.787641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875eed7-e877-43e1-90d8-2dfced75e5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:23.787912 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875eed7-e877-43e1-90d8-2dfced75e5cd admin admin] PATH_INFO: `/v3/projects/9ffe74cee98b482ab7cdda8a452c01e2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:23.789047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875eed7-e877-43e1-90d8-2dfced75e5cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ffe74cee98b482ab7cdda8a452c01e2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:23.791064 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875eed7-e877-43e1-90d8-2dfced75e5cd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:23.838452 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9875eed7-e877-43e1-90d8-2dfced75e5cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ffe74cee98b482ab7cdda8a452c01e2', 'request_id': 'req-9875eed7-e877-43e1-90d8-2dfced75e5cd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:23.846100 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9875eed7-e877-43e1-90d8-2dfced75e5cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ffe74cee98b482ab7cdda8a452c01e2', 'request_id': 'req-9875eed7-e877-43e1-90d8-2dfced75e5cd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:23.851714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9875eed7-e877-43e1-90d8-2dfced75e5cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ffe74cee98b482ab7cdda8a452c01e2', 'request_id': 'req-9875eed7-e877-43e1-90d8-2dfced75e5cd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:23.862842 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9875eed7-e877-43e1-90d8-2dfced75e5cd admin admin] The token cache is being invalidate because project 9ffe74cee98b482ab7cdda8a452c01e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:08:23.863125 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9875eed7-e877-43e1-90d8-2dfced75e5cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:23.863760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9875eed7-e877-43e1-90d8-2dfced75e5cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:23.865288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1049/2085] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:08:23 2022] DELETE /identity/v3/projects/9ffe74cee98b482ab7cdda8a452c01e2 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:23.870629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34407556-40d2-4b0f-a030-b5be630a3d72 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:23.870915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34407556-40d2-4b0f-a030-b5be630a3d72 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:23.871189 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34407556-40d2-4b0f-a030-b5be630a3d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:23.891130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-34407556-40d2-4b0f-a030-b5be630a3d72 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:23.892346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-34407556-40d2-4b0f-a030-b5be630a3d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:23.910376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:23.954347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1037/2086] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:23.962986 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4edf64-989e-4b5d-a7d0-84c0bbd3c744 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:23.963266 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4edf64-989e-4b5d-a7d0-84c0bbd3c744 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:23.963536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4edf64-989e-4b5d-a7d0-84c0bbd3c744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:23.982878 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9b4edf64-989e-4b5d-a7d0-84c0bbd3c744 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:23.984060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9b4edf64-989e-4b5d-a7d0-84c0bbd3c744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:23.991760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1050/2087] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:08:23.996278 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34407556-40d2-4b0f-a030-b5be630a3d72 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:24.006732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34407556-40d2-4b0f-a030-b5be630a3d72 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:24.011286 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568397a-dd31-4ee1-8b5f-45daf2bffff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:24.011888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568397a-dd31-4ee1-8b5f-45daf2bffff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:24.012169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568397a-dd31-4ee1-8b5f-45daf2bffff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:24.012442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568397a-dd31-4ee1-8b5f-45daf2bffff3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:24.013087 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568397a-dd31-4ee1-8b5f-45daf2bffff3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:08:24.013498 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568397a-dd31-4ee1-8b5f-45daf2bffff3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:24.015134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568397a-dd31-4ee1-8b5f-45daf2bffff3 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:24.027185 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1038/2088] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:08:24.031576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4edf64-989e-4b5d-a7d0-84c0bbd3c744 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:24.036014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:24.042947 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4edf64-989e-4b5d-a7d0-84c0bbd3c744 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:24.048486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e3d75-f8cb-4703-989d-11dc9c156e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:24.049082 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e3d75-f8cb-4703-989d-11dc9c156e26 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:24.049354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e3d75-f8cb-4703-989d-11dc9c156e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:24.049621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e3d75-f8cb-4703-989d-11dc9c156e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:24.050346 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e3d75-f8cb-4703-989d-11dc9c156e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:24.052231 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e3d75-f8cb-4703-989d-11dc9c156e26 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:24.080191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1051/2089] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:08:24 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:24.084340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568397a-dd31-4ee1-8b5f-45daf2bffff3 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:24.094398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568397a-dd31-4ee1-8b5f-45daf2bffff3 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:24.152616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e63de-17d6-4231-9457-8f0f685bf0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:24.153230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e63de-17d6-4231-9457-8f0f685bf0f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:24.153507 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e63de-17d6-4231-9457-8f0f685bf0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:24.153777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e63de-17d6-4231-9457-8f0f685bf0f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:24.154522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e63de-17d6-4231-9457-8f0f685bf0f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:24.156567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e63de-17d6-4231-9457-8f0f685bf0f5 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:24.178220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1039/2090] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:08:24 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:24.182384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e3d75-f8cb-4703-989d-11dc9c156e26 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:24.192561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e3d75-f8cb-4703-989d-11dc9c156e26 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:24.197784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864da20-f32c-4611-a347-d8f793766474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:24.198417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864da20-f32c-4611-a347-d8f793766474 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:24.198694 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864da20-f32c-4611-a347-d8f793766474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:24.198964 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864da20-f32c-4611-a347-d8f793766474 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:24.199606 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864da20-f32c-4611-a347-d8f793766474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:24.201188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864da20-f32c-4611-a347-d8f793766474 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:24.214204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1052/2091] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:08:24.215584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e63de-17d6-4231-9457-8f0f685bf0f5 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:24.225661 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e63de-17d6-4231-9457-8f0f685bf0f5 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:24.230384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c718db-f4e8-4ff2-a08c-cc637d5eb683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:24.230980 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c718db-f4e8-4ff2-a08c-cc637d5eb683 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:24.231257 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c718db-f4e8-4ff2-a08c-cc637d5eb683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:24.231525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c718db-f4e8-4ff2-a08c-cc637d5eb683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:24.232188 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c718db-f4e8-4ff2-a08c-cc637d5eb683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:24.233779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c718db-f4e8-4ff2-a08c-cc637d5eb683 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:24.245661 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1040/2092] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:08:24.249993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864da20-f32c-4611-a347-d8f793766474 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:24.260911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864da20-f32c-4611-a347-d8f793766474 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:24.265551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c11b36-bd9c-4507-8681-88cbea0f18f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:24.266215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c11b36-bd9c-4507-8681-88cbea0f18f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:24.266496 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c11b36-bd9c-4507-8681-88cbea0f18f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:24.266778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c11b36-bd9c-4507-8681-88cbea0f18f5 admin admin] PATH_INFO: `/v3/projects/7d6399206c2d4c3eb507de075fd03156/users/cbd3386b0e80446091fa3ceb8e40a8ea/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:24.281812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c11b36-bd9c-4507-8681-88cbea0f18f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d6399206c2d4c3eb507de075fd03156, user_id=cbd3386b0e80446091fa3ceb8e40a8ea, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:24.284568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c11b36-bd9c-4507-8681-88cbea0f18f5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:24.298998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1053/2093] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:08:24 2022] PUT /identity/v3/projects/7d6399206c2d4c3eb507de075fd03156/users/cbd3386b0e80446091fa3ceb8e40a8ea/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:24.315741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd1173-6568-41a7-a458-34ba04968448 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:24.315741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd1173-6568-41a7-a458-34ba04968448 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:24.315741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd1173-6568-41a7-a458-34ba04968448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:24.346265 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-83bd1173-6568-41a7-a458-34ba04968448 None None] MFA Rules not processed for user `cbd3386b0e80446091fa3ceb8e40a8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:24.347596 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-83bd1173-6568-41a7-a458-34ba04968448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:24.415299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1041/2094] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:24.426261 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c11b36-bd9c-4507-8681-88cbea0f18f5 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:24.428627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-33c11b36-bd9c-4507-8681-88cbea0f18f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:24.484751 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c11b36-bd9c-4507-8681-88cbea0f18f5 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:24.488461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c3f9c-52f9-4248-ab22-eb3cc8f9c726 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:24.489068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c3f9c-52f9-4248-ab22-eb3cc8f9c726 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:24.489345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c3f9c-52f9-4248-ab22-eb3cc8f9c726 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:24.489617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c3f9c-52f9-4248-ab22-eb3cc8f9c726 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:24.498869 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c3f9c-52f9-4248-ab22-eb3cc8f9c726 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:24.500273 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c3f9c-52f9-4248-ab22-eb3cc8f9c726 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:24.509493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1054/2095] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:08:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:08:31.343222 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7a9fb-f167-468b-88ba-eacf40c45ddf None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:31.343708 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7a9fb-f167-468b-88ba-eacf40c45ddf None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:31.343803 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7a9fb-f167-468b-88ba-eacf40c45ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:31.362421 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-bdf7a9fb-f167-468b-88ba-eacf40c45ddf None None] MFA Rules not processed for user `eef1afc3ece940248494501dd27e3920`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:31.363622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-bdf7a9fb-f167-468b-88ba-eacf40c45ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:31.375614 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:31.419124 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1042/2096] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:34.505018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=49,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:37.113063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c3f9c-52f9-4248-ab22-eb3cc8f9c726 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:37.178940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c3f9c-52f9-4248-ab22-eb3cc8f9c726 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:37.183680 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3facf4-5f43-4c6f-87e2-e5c4d7a4913f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:37.184386 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3facf4-5f43-4c6f-87e2-e5c4d7a4913f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:37.184666 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3facf4-5f43-4c6f-87e2-e5c4d7a4913f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:37.184939 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3facf4-5f43-4c6f-87e2-e5c4d7a4913f admin admin] PATH_INFO: `/v3/users/cbd3386b0e80446091fa3ceb8e40a8ea` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:37.186081 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3facf4-5f43-4c6f-87e2-e5c4d7a4913f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbd3386b0e80446091fa3ceb8e40a8ea)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:37.188033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3facf4-5f43-4c6f-87e2-e5c4d7a4913f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:37.247265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3b3facf4-5f43-4c6f-87e2-e5c4d7a4913f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbd3386b0e80446091fa3ceb8e40a8ea'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:37.247924 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3b3facf4-5f43-4c6f-87e2-e5c4d7a4913f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbd3386b0e80446091fa3ceb8e40a8ea'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:37.255199 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3b3facf4-5f43-4c6f-87e2-e5c4d7a4913f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbd3386b0e80446091fa3ceb8e40a8ea'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:37.264605 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3b3facf4-5f43-4c6f-87e2-e5c4d7a4913f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbd3386b0e80446091fa3ceb8e40a8ea'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:37.277327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3b3facf4-5f43-4c6f-87e2-e5c4d7a4913f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbd3386b0e80446091fa3ceb8e40a8ea'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:37.284281 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1055/2097] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:08:37 2022] DELETE /identity/v3/users/cbd3386b0e80446091fa3ceb8e40a8ea => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:08:37.288785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421b518-7177-479d-b292-8e2d74b440a3 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:37.289266 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421b518-7177-479d-b292-8e2d74b440a3 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:37.289548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421b518-7177-479d-b292-8e2d74b440a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:37.309134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-7421b518-7177-479d-b292-8e2d74b440a3 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:37.310479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7421b518-7177-479d-b292-8e2d74b440a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:37.374043 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1043/2098] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:37.384625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3facf4-5f43-4c6f-87e2-e5c4d7a4913f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:37.386062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3b3facf4-5f43-4c6f-87e2-e5c4d7a4913f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:37.462035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3facf4-5f43-4c6f-87e2-e5c4d7a4913f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:37.466484 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d50b4-80ac-4f3b-996c-af9de9e9a33d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:37.468665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d50b4-80ac-4f3b-996c-af9de9e9a33d service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:37.468665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d50b4-80ac-4f3b-996c-af9de9e9a33d service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:37.468665 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d50b4-80ac-4f3b-996c-af9de9e9a33d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:37.482235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d50b4-80ac-4f3b-996c-af9de9e9a33d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:37.484247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d50b4-80ac-4f3b-996c-af9de9e9a33d service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:37.497427 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1056/2099] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:08:37.915548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421b518-7177-479d-b292-8e2d74b440a3 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:37.917817 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7421b518-7177-479d-b292-8e2d74b440a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:37.937359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421b518-7177-479d-b292-8e2d74b440a3 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:37.945086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a705f3-721b-4bdb-ab83-2af58e502a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:37.946228 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a705f3-721b-4bdb-ab83-2af58e502a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:37.946740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a705f3-721b-4bdb-ab83-2af58e502a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:37.947235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a705f3-721b-4bdb-ab83-2af58e502a26 admin admin] PATH_INFO: `/v3/projects/7d6399206c2d4c3eb507de075fd03156` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:37.949192 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a705f3-721b-4bdb-ab83-2af58e502a26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d6399206c2d4c3eb507de075fd03156)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:37.952533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a705f3-721b-4bdb-ab83-2af58e502a26 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:38.008529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f8a705f3-721b-4bdb-ab83-2af58e502a26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d6399206c2d4c3eb507de075fd03156', 'request_id': 'req-f8a705f3-721b-4bdb-ab83-2af58e502a26'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:38.016523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f8a705f3-721b-4bdb-ab83-2af58e502a26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d6399206c2d4c3eb507de075fd03156', 'request_id': 'req-f8a705f3-721b-4bdb-ab83-2af58e502a26'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:38.022402 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f8a705f3-721b-4bdb-ab83-2af58e502a26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d6399206c2d4c3eb507de075fd03156', 'request_id': 'req-f8a705f3-721b-4bdb-ab83-2af58e502a26'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:38.033967 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f8a705f3-721b-4bdb-ab83-2af58e502a26 admin admin] The token cache is being invalidate because project 7d6399206c2d4c3eb507de075fd03156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:08:38.034359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f8a705f3-721b-4bdb-ab83-2af58e502a26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:38.035177 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f8a705f3-721b-4bdb-ab83-2af58e502a26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:38.036722 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1044/2100] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:08:37 2022] DELETE /identity/v3/projects/7d6399206c2d4c3eb507de075fd03156 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:38.042304 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf450d-a2cd-4c9d-92f1-87dd6f4441f8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:38.042593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf450d-a2cd-4c9d-92f1-87dd6f4441f8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:38.042894 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf450d-a2cd-4c9d-92f1-87dd6f4441f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:38.064417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5dbf450d-a2cd-4c9d-92f1-87dd6f4441f8 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:38.065778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5dbf450d-a2cd-4c9d-92f1-87dd6f4441f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:38.084619 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:38.176379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1057/2101] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:38.185137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9440a-9dc0-4ae0-ad5e-e5a11d1dc99d None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:38.185429 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9440a-9dc0-4ae0-ad5e-e5a11d1dc99d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:38.185709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9440a-9dc0-4ae0-ad5e-e5a11d1dc99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:38.209656 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-37a9440a-9dc0-4ae0-ad5e-e5a11d1dc99d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:38.211738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-37a9440a-9dc0-4ae0-ad5e-e5a11d1dc99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:38.232035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1045/2102] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:38.237019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf450d-a2cd-4c9d-92f1-87dd6f4441f8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:38.247670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf450d-a2cd-4c9d-92f1-87dd6f4441f8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:38.252794 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185759df-01ef-44f0-8290-77ce1717e467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:38.253463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185759df-01ef-44f0-8290-77ce1717e467 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:38.253587 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185759df-01ef-44f0-8290-77ce1717e467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:38.253803 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185759df-01ef-44f0-8290-77ce1717e467 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:38.254746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185759df-01ef-44f0-8290-77ce1717e467 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:08:38.255078 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185759df-01ef-44f0-8290-77ce1717e467 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:38.256605 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185759df-01ef-44f0-8290-77ce1717e467 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:38.273538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1058/2103] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:08:38.274778 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a9440a-9dc0-4ae0-ad5e-e5a11d1dc99d None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:38.285029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a9440a-9dc0-4ae0-ad5e-e5a11d1dc99d None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:38.290666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339a1f5-5ae1-4327-bbe9-00ec31a40a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:38.291849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339a1f5-5ae1-4327-bbe9-00ec31a40a82 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:38.292127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339a1f5-5ae1-4327-bbe9-00ec31a40a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:38.292397 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339a1f5-5ae1-4327-bbe9-00ec31a40a82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:38.293684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339a1f5-5ae1-4327-bbe9-00ec31a40a82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:38.295796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339a1f5-5ae1-4327-bbe9-00ec31a40a82 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:38.329216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1046/2104] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:08:38 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:38.333149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185759df-01ef-44f0-8290-77ce1717e467 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:38.344022 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185759df-01ef-44f0-8290-77ce1717e467 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:38.415261 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04425048-a947-4ff3-95d8-430260e26684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:38.416137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04425048-a947-4ff3-95d8-430260e26684 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:38.416511 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04425048-a947-4ff3-95d8-430260e26684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:38.416871 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04425048-a947-4ff3-95d8-430260e26684 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:38.417672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04425048-a947-4ff3-95d8-430260e26684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:38.419800 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04425048-a947-4ff3-95d8-430260e26684 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:38.451625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1059/2105] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:08:38 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:38.458552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339a1f5-5ae1-4327-bbe9-00ec31a40a82 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:38.472213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339a1f5-5ae1-4327-bbe9-00ec31a40a82 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:38.476971 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead30462-e113-4146-8c43-9259174bb82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:38.477577 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead30462-e113-4146-8c43-9259174bb82a admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:38.477855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead30462-e113-4146-8c43-9259174bb82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:38.478178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead30462-e113-4146-8c43-9259174bb82a admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:38.478829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead30462-e113-4146-8c43-9259174bb82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:38.480466 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead30462-e113-4146-8c43-9259174bb82a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:38.492738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1047/2106] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:08:38.496869 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04425048-a947-4ff3-95d8-430260e26684 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:38.507565 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04425048-a947-4ff3-95d8-430260e26684 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:38.512250 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425caa2-91d0-4061-97be-0963c0670dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:38.512845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425caa2-91d0-4061-97be-0963c0670dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:38.513128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425caa2-91d0-4061-97be-0963c0670dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:38.513401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425caa2-91d0-4061-97be-0963c0670dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:38.514046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425caa2-91d0-4061-97be-0963c0670dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:38.515774 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425caa2-91d0-4061-97be-0963c0670dc1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:38.526748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1060/2107] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:08:38.530827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead30462-e113-4146-8c43-9259174bb82a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:38.541246 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead30462-e113-4146-8c43-9259174bb82a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:38.547062 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3acc8-d152-4db3-a2bc-9b2c6edb2c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:38.547791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3acc8-d152-4db3-a2bc-9b2c6edb2c3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:38.548130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3acc8-d152-4db3-a2bc-9b2c6edb2c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:38.548462 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3acc8-d152-4db3-a2bc-9b2c6edb2c3a admin admin] PATH_INFO: `/v3/projects/ba28dcc4df9f457392f6ebd0697a0bce/users/de465fc86f45433bb8cb664d5fb9ccb5/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:38.551019 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:38.567104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3acc8-d152-4db3-a2bc-9b2c6edb2c3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba28dcc4df9f457392f6ebd0697a0bce, user_id=de465fc86f45433bb8cb664d5fb9ccb5, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:38.570553 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3acc8-d152-4db3-a2bc-9b2c6edb2c3a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:38.583722 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1048/2108] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:08:38 2022] PUT /identity/v3/projects/ba28dcc4df9f457392f6ebd0697a0bce/users/de465fc86f45433bb8cb664d5fb9ccb5/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:08:38.598087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84024584-70b8-4f24-90e5-c49e302dadca None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:38.598457 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84024584-70b8-4f24-90e5-c49e302dadca None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:38.598736 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84024584-70b8-4f24-90e5-c49e302dadca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:38.628686 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-84024584-70b8-4f24-90e5-c49e302dadca None None] MFA Rules not processed for user `de465fc86f45433bb8cb664d5fb9ccb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:38.629923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-84024584-70b8-4f24-90e5-c49e302dadca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:38.701971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1061/2109] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:38.711280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3acc8-d152-4db3-a2bc-9b2c6edb2c3a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:38.712982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ffd3acc8-d152-4db3-a2bc-9b2c6edb2c3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:38.773313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3acc8-d152-4db3-a2bc-9b2c6edb2c3a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:38.777676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aff5c7-4f12-4983-abb8-fca306a56a3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:38.778337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aff5c7-4f12-4983-abb8-fca306a56a3d service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:38.778699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aff5c7-4f12-4983-abb8-fca306a56a3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:38.779137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aff5c7-4f12-4983-abb8-fca306a56a3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:38.790250 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aff5c7-4f12-4983-abb8-fca306a56a3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:38.791714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aff5c7-4f12-4983-abb8-fca306a56a3d service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:38.802097 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1049/2110] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:08:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:08:38.829433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84024584-70b8-4f24-90e5-c49e302dadca None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:38.853605 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:38.903031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84024584-70b8-4f24-90e5-c49e302dadca None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:38.908088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36ec46-e5d4-4e0e-838f-c65ef1534562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:38.908772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36ec46-e5d4-4e0e-838f-c65ef1534562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:38.909048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36ec46-e5d4-4e0e-838f-c65ef1534562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:38.909340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36ec46-e5d4-4e0e-838f-c65ef1534562 admin admin] PATH_INFO: `/v3/users/de465fc86f45433bb8cb664d5fb9ccb5` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:38.910611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36ec46-e5d4-4e0e-838f-c65ef1534562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de465fc86f45433bb8cb664d5fb9ccb5)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:38.912599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36ec46-e5d4-4e0e-838f-c65ef1534562 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:38.963475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ab36ec46-e5d4-4e0e-838f-c65ef1534562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de465fc86f45433bb8cb664d5fb9ccb5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:38.964046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ab36ec46-e5d4-4e0e-838f-c65ef1534562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de465fc86f45433bb8cb664d5fb9ccb5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:38.971371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ab36ec46-e5d4-4e0e-838f-c65ef1534562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de465fc86f45433bb8cb664d5fb9ccb5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:38.981217 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ab36ec46-e5d4-4e0e-838f-c65ef1534562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de465fc86f45433bb8cb664d5fb9ccb5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:38.995528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ab36ec46-e5d4-4e0e-838f-c65ef1534562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de465fc86f45433bb8cb664d5fb9ccb5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:39.002577 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1062/2111] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:08:38 2022] DELETE /identity/v3/users/de465fc86f45433bb8cb664d5fb9ccb5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:39.006969 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f48b1-f0e9-471f-b42e-5f93d84ff755 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:39.007257 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f48b1-f0e9-471f-b42e-5f93d84ff755 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:39.007526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f48b1-f0e9-471f-b42e-5f93d84ff755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:39.027034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-684f48b1-f0e9-471f-b42e-5f93d84ff755 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:39.028228 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-684f48b1-f0e9-471f-b42e-5f93d84ff755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:39.089231 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1050/2112] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:08:39.098838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36ec46-e5d4-4e0e-838f-c65ef1534562 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:39.100357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ab36ec46-e5d4-4e0e-838f-c65ef1534562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:39.160053 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36ec46-e5d4-4e0e-838f-c65ef1534562 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:39.163868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099a865-e890-429a-9bf8-984e5fde45fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:39.164476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099a865-e890-429a-9bf8-984e5fde45fd service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:39.164755 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099a865-e890-429a-9bf8-984e5fde45fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:39.165025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099a865-e890-429a-9bf8-984e5fde45fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:39.174954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099a865-e890-429a-9bf8-984e5fde45fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:39.176386 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099a865-e890-429a-9bf8-984e5fde45fd service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:39.187067 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1063/2113] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:08:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:08:39.543982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f48b1-f0e9-471f-b42e-5f93d84ff755 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:39.545517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-684f48b1-f0e9-471f-b42e-5f93d84ff755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:39.567824 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f48b1-f0e9-471f-b42e-5f93d84ff755 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:39.573011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef3e49-f518-41d7-9580-ac9065395541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:39.573648 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef3e49-f518-41d7-9580-ac9065395541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:39.573928 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef3e49-f518-41d7-9580-ac9065395541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:39.574232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef3e49-f518-41d7-9580-ac9065395541 admin admin] PATH_INFO: `/v3/projects/ba28dcc4df9f457392f6ebd0697a0bce` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:39.575357 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef3e49-f518-41d7-9580-ac9065395541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba28dcc4df9f457392f6ebd0697a0bce)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:39.577269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef3e49-f518-41d7-9580-ac9065395541 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:39.624481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ffef3e49-f518-41d7-9580-ac9065395541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba28dcc4df9f457392f6ebd0697a0bce', 'request_id': 'req-ffef3e49-f518-41d7-9580-ac9065395541'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:39.633099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ffef3e49-f518-41d7-9580-ac9065395541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba28dcc4df9f457392f6ebd0697a0bce', 'request_id': 'req-ffef3e49-f518-41d7-9580-ac9065395541'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:39.639134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ffef3e49-f518-41d7-9580-ac9065395541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba28dcc4df9f457392f6ebd0697a0bce', 'request_id': 'req-ffef3e49-f518-41d7-9580-ac9065395541'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:39.651337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ffef3e49-f518-41d7-9580-ac9065395541 admin admin] The token cache is being invalidate because project ba28dcc4df9f457392f6ebd0697a0bce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:08:39.651905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ffef3e49-f518-41d7-9580-ac9065395541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:39.652515 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ffef3e49-f518-41d7-9580-ac9065395541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:39.653972 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1051/2114] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:08:39 2022] DELETE /identity/v3/projects/ba28dcc4df9f457392f6ebd0697a0bce => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:48.430133 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099a865-e890-429a-9bf8-984e5fde45fd service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:48.431549 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1099a865-e890-429a-9bf8-984e5fde45fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:48.444797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099a865-e890-429a-9bf8-984e5fde45fd service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:48.486778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:48.523915 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf478d4-f433-45b4-b78d-51487541b013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:48.524581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf478d4-f433-45b4-b78d-51487541b013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:48.524863 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf478d4-f433-45b4-b78d-51487541b013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:48.525136 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf478d4-f433-45b4-b78d-51487541b013 admin admin] PATH_INFO: `/v3/users/af3de82e9ce74d49a72cc5d14e5e3d67` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:48.526884 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf478d4-f433-45b4-b78d-51487541b013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3de82e9ce74d49a72cc5d14e5e3d67)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:48.528993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf478d4-f433-45b4-b78d-51487541b013 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:48.589482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ecf478d4-f433-45b4-b78d-51487541b013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3de82e9ce74d49a72cc5d14e5e3d67'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:48.590173 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ecf478d4-f433-45b4-b78d-51487541b013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3de82e9ce74d49a72cc5d14e5e3d67'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:48.599551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ecf478d4-f433-45b4-b78d-51487541b013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3de82e9ce74d49a72cc5d14e5e3d67'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:48.609838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ecf478d4-f433-45b4-b78d-51487541b013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3de82e9ce74d49a72cc5d14e5e3d67'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:48.636567 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ecf478d4-f433-45b4-b78d-51487541b013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3de82e9ce74d49a72cc5d14e5e3d67'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:48.643827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1064/2115] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:08:48 2022] DELETE /identity/v3/users/af3de82e9ce74d49a72cc5d14e5e3d67 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:48.649065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f8acb-2353-414d-8b98-3313d12895af None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:48.649786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f8acb-2353-414d-8b98-3313d12895af None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:48.651366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f8acb-2353-414d-8b98-3313d12895af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:48.674456 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a76f8acb-2353-414d-8b98-3313d12895af None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:48.675590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a76f8acb-2353-414d-8b98-3313d12895af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:48.680410 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:48.739201 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1052/2116] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:48.749660 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf478d4-f433-45b4-b78d-51487541b013 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:48.751093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ecf478d4-f433-45b4-b78d-51487541b013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:48.808957 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf478d4-f433-45b4-b78d-51487541b013 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:48.812553 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cebe14-0eff-4210-9aef-5ce66ea39ee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:48.813157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cebe14-0eff-4210-9aef-5ce66ea39ee9 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:48.813428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cebe14-0eff-4210-9aef-5ce66ea39ee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:48.813693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cebe14-0eff-4210-9aef-5ce66ea39ee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:48.822527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cebe14-0eff-4210-9aef-5ce66ea39ee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:48.823932 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cebe14-0eff-4210-9aef-5ce66ea39ee9 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:48.833090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1065/2117] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:08:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:08:49.094692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f8acb-2353-414d-8b98-3313d12895af None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:49.096116 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a76f8acb-2353-414d-8b98-3313d12895af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:49.108702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f8acb-2353-414d-8b98-3313d12895af None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:49.113424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bba25b-aa1f-45d0-ac9d-7aad8371569c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:49.114061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bba25b-aa1f-45d0-ac9d-7aad8371569c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:49.114379 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bba25b-aa1f-45d0-ac9d-7aad8371569c admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:49.114654 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bba25b-aa1f-45d0-ac9d-7aad8371569c admin admin] PATH_INFO: `/v3/projects/135fcd62ab6a49b8828c360cf95f41e3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:49.115817 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bba25b-aa1f-45d0-ac9d-7aad8371569c admin admin] RBAC: Authorizing `identity:delete_project(project_id=135fcd62ab6a49b8828c360cf95f41e3)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:49.117703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bba25b-aa1f-45d0-ac9d-7aad8371569c admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:49.166715 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-64bba25b-aa1f-45d0-ac9d-7aad8371569c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '135fcd62ab6a49b8828c360cf95f41e3', 'request_id': 'req-64bba25b-aa1f-45d0-ac9d-7aad8371569c'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:49.174461 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-64bba25b-aa1f-45d0-ac9d-7aad8371569c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '135fcd62ab6a49b8828c360cf95f41e3', 'request_id': 'req-64bba25b-aa1f-45d0-ac9d-7aad8371569c'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:49.180295 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-64bba25b-aa1f-45d0-ac9d-7aad8371569c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '135fcd62ab6a49b8828c360cf95f41e3', 'request_id': 'req-64bba25b-aa1f-45d0-ac9d-7aad8371569c'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:49.191196 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-64bba25b-aa1f-45d0-ac9d-7aad8371569c admin admin] The token cache is being invalidate because project 135fcd62ab6a49b8828c360cf95f41e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:08:49.191485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-64bba25b-aa1f-45d0-ac9d-7aad8371569c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:49.192142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-64bba25b-aa1f-45d0-ac9d-7aad8371569c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:49.193653 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1053/2118] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:08:49 2022] DELETE /identity/v3/projects/135fcd62ab6a49b8828c360cf95f41e3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:49.198747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cebe14-0eff-4210-9aef-5ce66ea39ee9 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:49.208658 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cebe14-0eff-4210-9aef-5ce66ea39ee9 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:49.265473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cf698-1cca-4a8b-b7ee-24e4b41a2f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:49.266097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cf698-1cca-4a8b-b7ee-24e4b41a2f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:49.266402 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cf698-1cca-4a8b-b7ee-24e4b41a2f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:49.266672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cf698-1cca-4a8b-b7ee-24e4b41a2f15 admin admin] PATH_INFO: `/v3/users/f745c28a6b3243518953bc8987e548ad` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:49.267776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12cf698-1cca-4a8b-b7ee-24e4b41a2f15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f745c28a6b3243518953bc8987e548ad)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:49.269741 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12cf698-1cca-4a8b-b7ee-24e4b41a2f15 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:49.272916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:49.321193 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f12cf698-1cca-4a8b-b7ee-24e4b41a2f15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f745c28a6b3243518953bc8987e548ad'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:49.321849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f12cf698-1cca-4a8b-b7ee-24e4b41a2f15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f745c28a6b3243518953bc8987e548ad'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:49.328688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f12cf698-1cca-4a8b-b7ee-24e4b41a2f15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f745c28a6b3243518953bc8987e548ad'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:49.335956 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f12cf698-1cca-4a8b-b7ee-24e4b41a2f15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f745c28a6b3243518953bc8987e548ad'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:49.348026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f12cf698-1cca-4a8b-b7ee-24e4b41a2f15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f745c28a6b3243518953bc8987e548ad'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:49.354541 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1066/2119] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:08:49 2022] DELETE /identity/v3/users/f745c28a6b3243518953bc8987e548ad => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:49.637638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bba25b-aa1f-45d0-ac9d-7aad8371569c admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:49.639198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-64bba25b-aa1f-45d0-ac9d-7aad8371569c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:49.655255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bba25b-aa1f-45d0-ac9d-7aad8371569c admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:49.727449 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b4212-6ebd-4905-9a25-2affe4152bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:49.728182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b4212-6ebd-4905-9a25-2affe4152bbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:49.728532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b4212-6ebd-4905-9a25-2affe4152bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:49.728810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b4212-6ebd-4905-9a25-2affe4152bbc admin admin] PATH_INFO: `/v3/projects/fc4b0757c98449039cf2656c0c5d36c8` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:49.729956 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b4212-6ebd-4905-9a25-2affe4152bbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc4b0757c98449039cf2656c0c5d36c8)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:49.731941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b4212-6ebd-4905-9a25-2affe4152bbc admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:49.776127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:49.791430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-672b4212-6ebd-4905-9a25-2affe4152bbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc4b0757c98449039cf2656c0c5d36c8', 'request_id': 'req-672b4212-6ebd-4905-9a25-2affe4152bbc'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:49.805383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-672b4212-6ebd-4905-9a25-2affe4152bbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc4b0757c98449039cf2656c0c5d36c8', 'request_id': 'req-672b4212-6ebd-4905-9a25-2affe4152bbc'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:49.815420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-672b4212-6ebd-4905-9a25-2affe4152bbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc4b0757c98449039cf2656c0c5d36c8', 'request_id': 'req-672b4212-6ebd-4905-9a25-2affe4152bbc'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:49.829752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-672b4212-6ebd-4905-9a25-2affe4152bbc admin admin] The token cache is being invalidate because project fc4b0757c98449039cf2656c0c5d36c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:08:49.830176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-672b4212-6ebd-4905-9a25-2affe4152bbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:49.831182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-672b4212-6ebd-4905-9a25-2affe4152bbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:49.837500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1054/2120] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:08:49 2022] DELETE /identity/v3/projects/fc4b0757c98449039cf2656c0c5d36c8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:49.839793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2523e9-f421-491d-b024-4b52b24b6a1e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:49.840604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2523e9-f421-491d-b024-4b52b24b6a1e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:49.841041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2523e9-f421-491d-b024-4b52b24b6a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:49.866017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-fd2523e9-f421-491d-b024-4b52b24b6a1e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:49.868226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fd2523e9-f421-491d-b024-4b52b24b6a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:49.964641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1067/2121] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:49.975094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b13fad-79c9-4f4e-9a3e-135e0ac2a49b None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:49.975483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b13fad-79c9-4f4e-9a3e-135e0ac2a49b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:49.975692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b13fad-79c9-4f4e-9a3e-135e0ac2a49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:50.013516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-31b13fad-79c9-4f4e-9a3e-135e0ac2a49b None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:50.014976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-31b13fad-79c9-4f4e-9a3e-135e0ac2a49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:50.024131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1055/2122] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:50.028371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2523e9-f421-491d-b024-4b52b24b6a1e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:50.038230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2523e9-f421-491d-b024-4b52b24b6a1e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:50.042979 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669fb9d-9512-4580-bd31-3e13f51bccca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:50.043481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669fb9d-9512-4580-bd31-3e13f51bccca admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:50.043754 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669fb9d-9512-4580-bd31-3e13f51bccca admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:50.044024 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669fb9d-9512-4580-bd31-3e13f51bccca admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:50.044677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669fb9d-9512-4580-bd31-3e13f51bccca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:08:50.045032 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669fb9d-9512-4580-bd31-3e13f51bccca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:50.046666 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669fb9d-9512-4580-bd31-3e13f51bccca admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:50.059020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1068/2123] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:08:50.063913 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b13fad-79c9-4f4e-9a3e-135e0ac2a49b None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:50.073899 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b13fad-79c9-4f4e-9a3e-135e0ac2a49b None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:50.078546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c83664-be3a-48da-a51b-63d42e03fd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:50.079138 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c83664-be3a-48da-a51b-63d42e03fd6f admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:50.079412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c83664-be3a-48da-a51b-63d42e03fd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:50.079674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c83664-be3a-48da-a51b-63d42e03fd6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:50.080304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c83664-be3a-48da-a51b-63d42e03fd6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:50.082216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c83664-be3a-48da-a51b-63d42e03fd6f admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:50.108597 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1056/2124] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:08:50 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:50.112963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669fb9d-9512-4580-bd31-3e13f51bccca admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:50.123260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669fb9d-9512-4580-bd31-3e13f51bccca admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:50.179835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81cc8a-9cd1-4c9b-b8a6-b373dcab3bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:50.180449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81cc8a-9cd1-4c9b-b8a6-b373dcab3bca admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:50.180726 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81cc8a-9cd1-4c9b-b8a6-b373dcab3bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:50.180997 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81cc8a-9cd1-4c9b-b8a6-b373dcab3bca admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:50.181657 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81cc8a-9cd1-4c9b-b8a6-b373dcab3bca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:50.183610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81cc8a-9cd1-4c9b-b8a6-b373dcab3bca admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:50.201301 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:50.208996 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1069/2125] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:08:50 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:50.213638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c83664-be3a-48da-a51b-63d42e03fd6f admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:50.224343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c83664-be3a-48da-a51b-63d42e03fd6f admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:50.229270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247b01b-348a-45eb-813e-5865223a8713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:50.229891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247b01b-348a-45eb-813e-5865223a8713 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:50.230213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247b01b-348a-45eb-813e-5865223a8713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:50.230480 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247b01b-348a-45eb-813e-5865223a8713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:50.231135 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247b01b-348a-45eb-813e-5865223a8713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:50.232723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247b01b-348a-45eb-813e-5865223a8713 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:50.244254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1057/2126] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:08:50.248613 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81cc8a-9cd1-4c9b-b8a6-b373dcab3bca admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:50.259239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81cc8a-9cd1-4c9b-b8a6-b373dcab3bca admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:50.263801 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba7b6a-00bc-44ac-8bf9-227eed9b3dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:50.264395 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba7b6a-00bc-44ac-8bf9-227eed9b3dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:50.264673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba7b6a-00bc-44ac-8bf9-227eed9b3dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:50.264946 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba7b6a-00bc-44ac-8bf9-227eed9b3dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:50.265586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba7b6a-00bc-44ac-8bf9-227eed9b3dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:50.267205 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba7b6a-00bc-44ac-8bf9-227eed9b3dbf admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:50.277698 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1070/2127] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:08:50.281789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247b01b-348a-45eb-813e-5865223a8713 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:50.291643 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247b01b-348a-45eb-813e-5865223a8713 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:50.296206 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810820c-d0b9-4dbc-97d6-ac4dbeca9266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:50.296845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810820c-d0b9-4dbc-97d6-ac4dbeca9266 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:50.297122 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810820c-d0b9-4dbc-97d6-ac4dbeca9266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:50.297394 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810820c-d0b9-4dbc-97d6-ac4dbeca9266 admin admin] PATH_INFO: `/v3/projects/f6cc38b7e88a48c082fdde55ec15d726/users/c380e971c1ed4212b2fbbe767c2307a7/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:50.312329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810820c-d0b9-4dbc-97d6-ac4dbeca9266 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6cc38b7e88a48c082fdde55ec15d726, user_id=c380e971c1ed4212b2fbbe767c2307a7, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:50.315048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810820c-d0b9-4dbc-97d6-ac4dbeca9266 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:50.331572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1058/2128] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:08:50 2022] PUT /identity/v3/projects/f6cc38b7e88a48c082fdde55ec15d726/users/c380e971c1ed4212b2fbbe767c2307a7/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:50.342140 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989c1b9-9f01-4e8e-ab4c-b239f79e9c2b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:50.342443 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989c1b9-9f01-4e8e-ab4c-b239f79e9c2b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:50.342714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989c1b9-9f01-4e8e-ab4c-b239f79e9c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:50.371225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7989c1b9-9f01-4e8e-ab4c-b239f79e9c2b None None] MFA Rules not processed for user `c380e971c1ed4212b2fbbe767c2307a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:50.372439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7989c1b9-9f01-4e8e-ab4c-b239f79e9c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:50.435888 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1071/2129] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:50 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:50.440405 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810820c-d0b9-4dbc-97d6-ac4dbeca9266 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:50.502620 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810820c-d0b9-4dbc-97d6-ac4dbeca9266 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:50.507751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4cbe4-e3d0-4cda-a5f0-6e2163b65cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:50.508348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4cbe4-e3d0-4cda-a5f0-6e2163b65cdf admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:50.508626 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4cbe4-e3d0-4cda-a5f0-6e2163b65cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:50.508895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4cbe4-e3d0-4cda-a5f0-6e2163b65cdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:50.509561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4cbe4-e3d0-4cda-a5f0-6e2163b65cdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:50.511495 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4cbe4-e3d0-4cda-a5f0-6e2163b65cdf admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:50.540319 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1059/2130] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:08:50 2022] POST /identity/v3/projects => generated 401 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:50.544437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989c1b9-9f01-4e8e-ab4c-b239f79e9c2b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:50.555497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989c1b9-9f01-4e8e-ab4c-b239f79e9c2b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:50.611487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d81cb-a4aa-465e-a04d-3aeb7dc3530e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:50.612097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d81cb-a4aa-465e-a04d-3aeb7dc3530e admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:50.612375 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d81cb-a4aa-465e-a04d-3aeb7dc3530e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:50.612645 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d81cb-a4aa-465e-a04d-3aeb7dc3530e admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:50.613330 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d81cb-a4aa-465e-a04d-3aeb7dc3530e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:50.615204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d81cb-a4aa-465e-a04d-3aeb7dc3530e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:50.637721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1072/2131] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:08:50 2022] POST /identity/v3/users => generated 307 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:50.641807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4cbe4-e3d0-4cda-a5f0-6e2163b65cdf admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:50.652643 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4cbe4-e3d0-4cda-a5f0-6e2163b65cdf admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:50.657356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8c88e-3764-43f8-9347-7f5397111163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:50.657938 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8c88e-3764-43f8-9347-7f5397111163 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:50.658253 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8c88e-3764-43f8-9347-7f5397111163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:50.658523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8c88e-3764-43f8-9347-7f5397111163 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:50.659156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8c88e-3764-43f8-9347-7f5397111163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:50.660766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8c88e-3764-43f8-9347-7f5397111163 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:50.673989 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1060/2132] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:08:50.679236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d81cb-a4aa-465e-a04d-3aeb7dc3530e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:50.689149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d81cb-a4aa-465e-a04d-3aeb7dc3530e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:50.694006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12152e-e63b-4e53-8dd2-d3aa1c591da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:50.694889 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12152e-e63b-4e53-8dd2-d3aa1c591da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:50.695163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12152e-e63b-4e53-8dd2-d3aa1c591da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:50.695432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12152e-e63b-4e53-8dd2-d3aa1c591da4 admin admin] PATH_INFO: `/v3/projects/4f58a9b16fa645c189e71ba3f37e2c65/users/b3850f4fdffb4d1f8d4381f51b3dee5d/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:50.710696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12152e-e63b-4e53-8dd2-d3aa1c591da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f58a9b16fa645c189e71ba3f37e2c65, user_id=b3850f4fdffb4d1f8d4381f51b3dee5d, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:50.713454 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12152e-e63b-4e53-8dd2-d3aa1c591da4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:50.727377 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1073/2133] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:08:50 2022] PUT /identity/v3/projects/4f58a9b16fa645c189e71ba3f37e2c65/users/b3850f4fdffb4d1f8d4381f51b3dee5d/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:08:50.731376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8c88e-3764-43f8-9347-7f5397111163 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:50.741889 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8c88e-3764-43f8-9347-7f5397111163 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:50.770605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:50.801273 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec0c45-a36b-46f9-ac11-83234d47b84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:50.801888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec0c45-a36b-46f9-ac11-83234d47b84d admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:50.802265 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec0c45-a36b-46f9-ac11-83234d47b84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:50.802543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec0c45-a36b-46f9-ac11-83234d47b84d admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:50.803196 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec0c45-a36b-46f9-ac11-83234d47b84d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:50.805387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec0c45-a36b-46f9-ac11-83234d47b84d admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:50.815528 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1061/2134] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:08:50.819796 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12152e-e63b-4e53-8dd2-d3aa1c591da4 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:50.830327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12152e-e63b-4e53-8dd2-d3aa1c591da4 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:50.835039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55e0be-5d48-4b55-a566-9192a8f4088d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:50.835747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55e0be-5d48-4b55-a566-9192a8f4088d admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:50.835987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55e0be-5d48-4b55-a566-9192a8f4088d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:50.836255 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55e0be-5d48-4b55-a566-9192a8f4088d admin admin] PATH_INFO: `/v3/projects/4f58a9b16fa645c189e71ba3f37e2c65/users/b3850f4fdffb4d1f8d4381f51b3dee5d/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:50.838172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55e0be-5d48-4b55-a566-9192a8f4088d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f58a9b16fa645c189e71ba3f37e2c65, user_id=b3850f4fdffb4d1f8d4381f51b3dee5d, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:50.840800 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55e0be-5d48-4b55-a566-9192a8f4088d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:50.852782 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1074/2135] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:08:50 2022] PUT /identity/v3/projects/4f58a9b16fa645c189e71ba3f37e2c65/users/b3850f4fdffb4d1f8d4381f51b3dee5d/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:50.863369 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28a5ea-2aa1-4876-bf13-d302ae42a5d9 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:50.863664 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28a5ea-2aa1-4876-bf13-d302ae42a5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:50.863940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28a5ea-2aa1-4876-bf13-d302ae42a5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:50.893367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-0e28a5ea-2aa1-4876-bf13-d302ae42a5d9 None None] MFA Rules not processed for user `b3850f4fdffb4d1f8d4381f51b3dee5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:50.894677 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0e28a5ea-2aa1-4876-bf13-d302ae42a5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:50.966981 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1062/2136] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:50 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:50.978597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55e0be-5d48-4b55-a566-9192a8f4088d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:50.980553 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7c55e0be-5d48-4b55-a566-9192a8f4088d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:51.014482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:51.043702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55e0be-5d48-4b55-a566-9192a8f4088d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:51.048251 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a5351-7163-41e7-9b40-deca468f3f77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:51.048894 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a5351-7163-41e7-9b40-deca468f3f77 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:51.049309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a5351-7163-41e7-9b40-deca468f3f77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:51.049585 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a5351-7163-41e7-9b40-deca468f3f77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:51.062183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a5351-7163-41e7-9b40-deca468f3f77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:51.071190 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a5351-7163-41e7-9b40-deca468f3f77 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:51.082014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1075/2137] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:08:50 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:08:51.174348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28a5ea-2aa1-4876-bf13-d302ae42a5d9 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:51.184077 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28a5ea-2aa1-4876-bf13-d302ae42a5d9 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:51.188016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35790b-c95b-4fc9-9f55-d9a4f3bbdb56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:51.188694 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35790b-c95b-4fc9-9f55-d9a4f3bbdb56 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:51.189119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35790b-c95b-4fc9-9f55-d9a4f3bbdb56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:51.189453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35790b-c95b-4fc9-9f55-d9a4f3bbdb56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:51.248268 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35790b-c95b-4fc9-9f55-d9a4f3bbdb56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:51.249705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35790b-c95b-4fc9-9f55-d9a4f3bbdb56 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:51.258521 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1063/2138] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:08:51 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:08:55.348798 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a5351-7163-41e7-9b40-deca468f3f77 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:55.413521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a5351-7163-41e7-9b40-deca468f3f77 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:55.418439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aacca-82b2-4529-9f89-4170266c146b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:55.419072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aacca-82b2-4529-9f89-4170266c146b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:55.419401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aacca-82b2-4529-9f89-4170266c146b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:55.419680 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aacca-82b2-4529-9f89-4170266c146b admin admin] PATH_INFO: `/v3/users/c380e971c1ed4212b2fbbe767c2307a7` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:55.420842 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180aacca-82b2-4529-9f89-4170266c146b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c380e971c1ed4212b2fbbe767c2307a7)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:55.422831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180aacca-82b2-4529-9f89-4170266c146b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:55.471812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-180aacca-82b2-4529-9f89-4170266c146b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c380e971c1ed4212b2fbbe767c2307a7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:55.472764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-180aacca-82b2-4529-9f89-4170266c146b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c380e971c1ed4212b2fbbe767c2307a7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:55.480717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-180aacca-82b2-4529-9f89-4170266c146b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c380e971c1ed4212b2fbbe767c2307a7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:55.489778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-180aacca-82b2-4529-9f89-4170266c146b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c380e971c1ed4212b2fbbe767c2307a7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:55.504344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-180aacca-82b2-4529-9f89-4170266c146b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c380e971c1ed4212b2fbbe767c2307a7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:55.511015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1076/2139] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:08:55 2022] DELETE /identity/v3/users/c380e971c1ed4212b2fbbe767c2307a7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:55.515532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c7c3e-e23e-4e21-a60d-257d248bdc88 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:55.516042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c7c3e-e23e-4e21-a60d-257d248bdc88 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:55.516144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c7c3e-e23e-4e21-a60d-257d248bdc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:55.537250 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-fe8c7c3e-e23e-4e21-a60d-257d248bdc88 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:55.539054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-fe8c7c3e-e23e-4e21-a60d-257d248bdc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:55.594106 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:55.607182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1064/2140] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:55.617446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aacca-82b2-4529-9f89-4170266c146b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:55.618870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-180aacca-82b2-4529-9f89-4170266c146b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:55.675395 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aacca-82b2-4529-9f89-4170266c146b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:55.679611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121599c7-6df6-4362-a3c2-dd5d30e4cff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:55.680212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121599c7-6df6-4362-a3c2-dd5d30e4cff2 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:55.680478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121599c7-6df6-4362-a3c2-dd5d30e4cff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:55.680748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121599c7-6df6-4362-a3c2-dd5d30e4cff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:55.686970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:55.692217 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121599c7-6df6-4362-a3c2-dd5d30e4cff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:55.693775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121599c7-6df6-4362-a3c2-dd5d30e4cff2 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:55.705825 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1077/2141] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:08:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:08:55.977933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c7c3e-e23e-4e21-a60d-257d248bdc88 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:55.979966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-fe8c7c3e-e23e-4e21-a60d-257d248bdc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:55.994147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c7c3e-e23e-4e21-a60d-257d248bdc88 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:55.999236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fedd05-0286-420e-9d3c-1b52abb273ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:55.999874 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fedd05-0286-420e-9d3c-1b52abb273ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:56.000152 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fedd05-0286-420e-9d3c-1b52abb273ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:56.000425 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fedd05-0286-420e-9d3c-1b52abb273ca admin admin] PATH_INFO: `/v3/projects/f6cc38b7e88a48c082fdde55ec15d726` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:56.001573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fedd05-0286-420e-9d3c-1b52abb273ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6cc38b7e88a48c082fdde55ec15d726)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:56.003523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fedd05-0286-420e-9d3c-1b52abb273ca admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:56.051532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-49fedd05-0286-420e-9d3c-1b52abb273ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6cc38b7e88a48c082fdde55ec15d726', 'request_id': 'req-49fedd05-0286-420e-9d3c-1b52abb273ca'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:56.059699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-49fedd05-0286-420e-9d3c-1b52abb273ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6cc38b7e88a48c082fdde55ec15d726', 'request_id': 'req-49fedd05-0286-420e-9d3c-1b52abb273ca'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:56.066966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-49fedd05-0286-420e-9d3c-1b52abb273ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6cc38b7e88a48c082fdde55ec15d726', 'request_id': 'req-49fedd05-0286-420e-9d3c-1b52abb273ca'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:56.081058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-49fedd05-0286-420e-9d3c-1b52abb273ca admin admin] The token cache is being invalidate because project f6cc38b7e88a48c082fdde55ec15d726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:08:56.081527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-49fedd05-0286-420e-9d3c-1b52abb273ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:56.082246 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-49fedd05-0286-420e-9d3c-1b52abb273ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:56.083752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1065/2142] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:08:55 2022] DELETE /identity/v3/projects/f6cc38b7e88a48c082fdde55ec15d726 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:56.088324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121599c7-6df6-4362-a3c2-dd5d30e4cff2 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:56.097787 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121599c7-6df6-4362-a3c2-dd5d30e4cff2 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:56.155354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3eb68-adbe-4759-bf0e-1029a67f9bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:56.156054 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3eb68-adbe-4759-bf0e-1029a67f9bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:56.156269 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3eb68-adbe-4759-bf0e-1029a67f9bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:56.156543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3eb68-adbe-4759-bf0e-1029a67f9bde admin admin] PATH_INFO: `/v3/users/b3850f4fdffb4d1f8d4381f51b3dee5d` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:56.157717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3eb68-adbe-4759-bf0e-1029a67f9bde admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3850f4fdffb4d1f8d4381f51b3dee5d)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:56.159674 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3eb68-adbe-4759-bf0e-1029a67f9bde admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:56.209644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-33e3eb68-adbe-4759-bf0e-1029a67f9bde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3850f4fdffb4d1f8d4381f51b3dee5d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:56.210837 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-33e3eb68-adbe-4759-bf0e-1029a67f9bde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3850f4fdffb4d1f8d4381f51b3dee5d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:56.218173 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-33e3eb68-adbe-4759-bf0e-1029a67f9bde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3850f4fdffb4d1f8d4381f51b3dee5d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:56.230218 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-33e3eb68-adbe-4759-bf0e-1029a67f9bde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3850f4fdffb4d1f8d4381f51b3dee5d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:56.244254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-33e3eb68-adbe-4759-bf0e-1029a67f9bde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3850f4fdffb4d1f8d4381f51b3dee5d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:56.251198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1078/2143] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:08:56 2022] DELETE /identity/v3/users/b3850f4fdffb4d1f8d4381f51b3dee5d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:56.514248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fedd05-0286-420e-9d3c-1b52abb273ca admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:56.515683 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-49fedd05-0286-420e-9d3c-1b52abb273ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:56.529224 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fedd05-0286-420e-9d3c-1b52abb273ca admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:56.591885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b252dca-452a-41bc-a6a4-42152818dc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:56.592516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b252dca-452a-41bc-a6a4-42152818dc22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:56.592789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b252dca-452a-41bc-a6a4-42152818dc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:56.593055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b252dca-452a-41bc-a6a4-42152818dc22 admin admin] PATH_INFO: `/v3/projects/4f58a9b16fa645c189e71ba3f37e2c65` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:56.594208 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b252dca-452a-41bc-a6a4-42152818dc22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f58a9b16fa645c189e71ba3f37e2c65)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:56.596096 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b252dca-452a-41bc-a6a4-42152818dc22 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:56.643474 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2b252dca-452a-41bc-a6a4-42152818dc22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f58a9b16fa645c189e71ba3f37e2c65', 'request_id': 'req-2b252dca-452a-41bc-a6a4-42152818dc22'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:56.652526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2b252dca-452a-41bc-a6a4-42152818dc22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f58a9b16fa645c189e71ba3f37e2c65', 'request_id': 'req-2b252dca-452a-41bc-a6a4-42152818dc22'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:56.659480 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2b252dca-452a-41bc-a6a4-42152818dc22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f58a9b16fa645c189e71ba3f37e2c65', 'request_id': 'req-2b252dca-452a-41bc-a6a4-42152818dc22'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:56.671532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2b252dca-452a-41bc-a6a4-42152818dc22 admin admin] The token cache is being invalidate because project 4f58a9b16fa645c189e71ba3f37e2c65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:08:56.671823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2b252dca-452a-41bc-a6a4-42152818dc22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:56.672540 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2b252dca-452a-41bc-a6a4-42152818dc22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:08:56.674313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1066/2144] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:08:56 2022] DELETE /identity/v3/projects/4f58a9b16fa645c189e71ba3f37e2c65 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:56.679535 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21de5a-230b-4fb4-b9e1-04a921780437 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:56.680018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21de5a-230b-4fb4-b9e1-04a921780437 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:56.680110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21de5a-230b-4fb4-b9e1-04a921780437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:56.699750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9d21de5a-230b-4fb4-b9e1-04a921780437 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:56.700956 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9d21de5a-230b-4fb4-b9e1-04a921780437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:56.764252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1079/2145] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:56.773466 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598eeae8-60f9-4c28-aac3-50ba11d5266b None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:56.773754 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598eeae8-60f9-4c28-aac3-50ba11d5266b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:56.774030 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598eeae8-60f9-4c28-aac3-50ba11d5266b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:56.788944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:56.794809 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-598eeae8-60f9-4c28-aac3-50ba11d5266b None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:56.796392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-598eeae8-60f9-4c28-aac3-50ba11d5266b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:56.804556 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1067/2146] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:56.808861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21de5a-230b-4fb4-b9e1-04a921780437 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:56.818493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21de5a-230b-4fb4-b9e1-04a921780437 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:56.823067 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2d2b0-82b6-43a0-8991-ce89a526f60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:56.823662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2d2b0-82b6-43a0-8991-ce89a526f60d admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:56.823937 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2d2b0-82b6-43a0-8991-ce89a526f60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:56.824207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2d2b0-82b6-43a0-8991-ce89a526f60d admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:56.824848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2d2b0-82b6-43a0-8991-ce89a526f60d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:08:56.825147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2d2b0-82b6-43a0-8991-ce89a526f60d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:56.826767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2d2b0-82b6-43a0-8991-ce89a526f60d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:56.836146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:56.839503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1080/2147] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:08:56.844019 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598eeae8-60f9-4c28-aac3-50ba11d5266b None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:56.856992 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598eeae8-60f9-4c28-aac3-50ba11d5266b None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:56.862069 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721438cc-694e-46a7-907b-bedb54c22300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:56.862688 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721438cc-694e-46a7-907b-bedb54c22300 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:56.862995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721438cc-694e-46a7-907b-bedb54c22300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:56.863269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721438cc-694e-46a7-907b-bedb54c22300 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:56.863894 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721438cc-694e-46a7-907b-bedb54c22300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:56.865811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721438cc-694e-46a7-907b-bedb54c22300 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:56.894875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1068/2148] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:08:56 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:56.899720 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2d2b0-82b6-43a0-8991-ce89a526f60d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:56.909646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2d2b0-82b6-43a0-8991-ce89a526f60d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:56.965730 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92e89c-66ba-4400-9d2a-a5bc0662d217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:56.966355 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92e89c-66ba-4400-9d2a-a5bc0662d217 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:56.966632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92e89c-66ba-4400-9d2a-a5bc0662d217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:56.966923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92e89c-66ba-4400-9d2a-a5bc0662d217 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:56.967541 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92e89c-66ba-4400-9d2a-a5bc0662d217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:56.969422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92e89c-66ba-4400-9d2a-a5bc0662d217 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:56.991358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1081/2149] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:08:56 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:56.995662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721438cc-694e-46a7-907b-bedb54c22300 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:57.005979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721438cc-694e-46a7-907b-bedb54c22300 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:57.011162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d795ae-1ae0-42ff-b6a9-12c3529998db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:57.011758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d795ae-1ae0-42ff-b6a9-12c3529998db admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:57.012067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d795ae-1ae0-42ff-b6a9-12c3529998db admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:57.012343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d795ae-1ae0-42ff-b6a9-12c3529998db admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:57.012983 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d795ae-1ae0-42ff-b6a9-12c3529998db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:57.014664 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d795ae-1ae0-42ff-b6a9-12c3529998db admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:57.025067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1069/2150] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:08:57.029048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92e89c-66ba-4400-9d2a-a5bc0662d217 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:57.039481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92e89c-66ba-4400-9d2a-a5bc0662d217 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:57.044585 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afb069-1d2b-44b7-b2cc-f9fa8e7dbe28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:57.045180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afb069-1d2b-44b7-b2cc-f9fa8e7dbe28 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:57.045468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afb069-1d2b-44b7-b2cc-f9fa8e7dbe28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:57.045741 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afb069-1d2b-44b7-b2cc-f9fa8e7dbe28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:57.046443 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afb069-1d2b-44b7-b2cc-f9fa8e7dbe28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:57.048024 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afb069-1d2b-44b7-b2cc-f9fa8e7dbe28 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:57.059432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1082/2151] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:08:57.063610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d795ae-1ae0-42ff-b6a9-12c3529998db admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:57.073804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d795ae-1ae0-42ff-b6a9-12c3529998db admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:57.078598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8256b8-2153-4ea6-8f8c-3dea35d6286f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:57.079278 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8256b8-2153-4ea6-8f8c-3dea35d6286f admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:57.079559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8256b8-2153-4ea6-8f8c-3dea35d6286f admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:57.079834 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8256b8-2153-4ea6-8f8c-3dea35d6286f admin admin] PATH_INFO: `/v3/projects/c0474b10d5104ac7a38196e80bfc2dfb/users/33e2f1bc844b4a259ab69b629888b244/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:57.095148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8256b8-2153-4ea6-8f8c-3dea35d6286f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0474b10d5104ac7a38196e80bfc2dfb, user_id=33e2f1bc844b4a259ab69b629888b244, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:57.097799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8256b8-2153-4ea6-8f8c-3dea35d6286f admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:57.112282 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1070/2152] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:08:57 2022] PUT /identity/v3/projects/c0474b10d5104ac7a38196e80bfc2dfb/users/33e2f1bc844b4a259ab69b629888b244/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:57.123343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c412e3f-1c6f-4317-b417-e18cd9eb4d51 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:57.123624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c412e3f-1c6f-4317-b417-e18cd9eb4d51 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:57.123895 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c412e3f-1c6f-4317-b417-e18cd9eb4d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:57.151711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-6c412e3f-1c6f-4317-b417-e18cd9eb4d51 None None] MFA Rules not processed for user `33e2f1bc844b4a259ab69b629888b244`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:57.152902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6c412e3f-1c6f-4317-b417-e18cd9eb4d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:57.217003 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1083/2153] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:57.222071 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8256b8-2153-4ea6-8f8c-3dea35d6286f admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:57.288018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8256b8-2153-4ea6-8f8c-3dea35d6286f admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:57.293311 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b94b0b-ba45-4d63-87fc-702529fa1f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:57.293920 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b94b0b-ba45-4d63-87fc-702529fa1f9d admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:57.294228 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b94b0b-ba45-4d63-87fc-702529fa1f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:57.294522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b94b0b-ba45-4d63-87fc-702529fa1f9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:57.295152 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b94b0b-ba45-4d63-87fc-702529fa1f9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:57.297063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b94b0b-ba45-4d63-87fc-702529fa1f9d admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:57.336484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1071/2154] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:08:57 2022] POST /identity/v3/projects => generated 367 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:57.340568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c412e3f-1c6f-4317-b417-e18cd9eb4d51 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:57.352227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c412e3f-1c6f-4317-b417-e18cd9eb4d51 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:57.416314 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5549f10-effe-486e-9704-21561fee569e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:57.416962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5549f10-effe-486e-9704-21561fee569e admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:57.417267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5549f10-effe-486e-9704-21561fee569e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:57.417576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5549f10-effe-486e-9704-21561fee569e admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:57.418253 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5549f10-effe-486e-9704-21561fee569e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:57.420219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5549f10-effe-486e-9704-21561fee569e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:57.436638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:57.444260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1084/2155] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:08:57 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:08:57.448291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b94b0b-ba45-4d63-87fc-702529fa1f9d admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:57.458622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b94b0b-ba45-4d63-87fc-702529fa1f9d admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:57.463986 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad368e50-a83c-441e-a7ff-15d53c1eebd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:57.464590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad368e50-a83c-441e-a7ff-15d53c1eebd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:57.464862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad368e50-a83c-441e-a7ff-15d53c1eebd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:57.465128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad368e50-a83c-441e-a7ff-15d53c1eebd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:57.465777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad368e50-a83c-441e-a7ff-15d53c1eebd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:57.467404 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad368e50-a83c-441e-a7ff-15d53c1eebd9 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:57.479001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1072/2156] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:08:57.483108 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5549f10-effe-486e-9704-21561fee569e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:57.493333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5549f10-effe-486e-9704-21561fee569e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:57.498662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73cf45f-993a-4583-a2ea-2fb5ac689ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:57.499309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73cf45f-993a-4583-a2ea-2fb5ac689ede admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:57.499606 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73cf45f-993a-4583-a2ea-2fb5ac689ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:57.499879 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73cf45f-993a-4583-a2ea-2fb5ac689ede admin admin] PATH_INFO: `/v3/projects/4c723ba5d5bd42e28cb9e968dc1d3cb4/users/25f4e995dea147cb87f1e7fd832be438/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:57.514446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73cf45f-993a-4583-a2ea-2fb5ac689ede admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c723ba5d5bd42e28cb9e968dc1d3cb4, user_id=25f4e995dea147cb87f1e7fd832be438, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:57.517097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73cf45f-993a-4583-a2ea-2fb5ac689ede admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:57.532265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1085/2157] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:08:57 2022] PUT /identity/v3/projects/4c723ba5d5bd42e28cb9e968dc1d3cb4/users/25f4e995dea147cb87f1e7fd832be438/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:08:57.536504 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad368e50-a83c-441e-a7ff-15d53c1eebd9 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:57.547297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad368e50-a83c-441e-a7ff-15d53c1eebd9 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:57.603871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c7a2f-ff22-4b4c-85ed-7b2def2c37ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:57.604480 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c7a2f-ff22-4b4c-85ed-7b2def2c37ea admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:57.604763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c7a2f-ff22-4b4c-85ed-7b2def2c37ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:57.605036 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c7a2f-ff22-4b4c-85ed-7b2def2c37ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:57.605685 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c7a2f-ff22-4b4c-85ed-7b2def2c37ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:57.607325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c7a2f-ff22-4b4c-85ed-7b2def2c37ea admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:57.618357 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1073/2158] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:08:57.622827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73cf45f-993a-4583-a2ea-2fb5ac689ede admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:57.633223 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73cf45f-993a-4583-a2ea-2fb5ac689ede admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:57.637792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedf92b-2a4d-4a51-a07f-6c336956b1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:57.638474 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedf92b-2a4d-4a51-a07f-6c336956b1f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:57.638756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedf92b-2a4d-4a51-a07f-6c336956b1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:57.639024 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedf92b-2a4d-4a51-a07f-6c336956b1f5 admin admin] PATH_INFO: `/v3/projects/4c723ba5d5bd42e28cb9e968dc1d3cb4/users/25f4e995dea147cb87f1e7fd832be438/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:57.640943 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedf92b-2a4d-4a51-a07f-6c336956b1f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c723ba5d5bd42e28cb9e968dc1d3cb4, user_id=25f4e995dea147cb87f1e7fd832be438, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:57.643586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedf92b-2a4d-4a51-a07f-6c336956b1f5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:57.654355 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1086/2159] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:08:57 2022] PUT /identity/v3/projects/4c723ba5d5bd42e28cb9e968dc1d3cb4/users/25f4e995dea147cb87f1e7fd832be438/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:08:57.666049 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ef930-5214-4f9d-b5ec-adc098d0c9c8 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:57.666370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ef930-5214-4f9d-b5ec-adc098d0c9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:57.666672 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ef930-5214-4f9d-b5ec-adc098d0c9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:57.692777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:57.696436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-230ef930-5214-4f9d-b5ec-adc098d0c9c8 None None] MFA Rules not processed for user `25f4e995dea147cb87f1e7fd832be438`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:57.698247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-230ef930-5214-4f9d-b5ec-adc098d0c9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:57.768736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1074/2160] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:08:57 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:57.780423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedf92b-2a4d-4a51-a07f-6c336956b1f5 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:57.782207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bdedf92b-2a4d-4a51-a07f-6c336956b1f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:57.843098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedf92b-2a4d-4a51-a07f-6c336956b1f5 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:57.846726 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473fb0d-208d-4398-b6e4-1d0c52e56979 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:57.847710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473fb0d-208d-4398-b6e4-1d0c52e56979 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:57.847710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473fb0d-208d-4398-b6e4-1d0c52e56979 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:57.847924 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473fb0d-208d-4398-b6e4-1d0c52e56979 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:57.857384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473fb0d-208d-4398-b6e4-1d0c52e56979 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:57.858831 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473fb0d-208d-4398-b6e4-1d0c52e56979 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:57.868249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1087/2161] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:08:57 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:08:57.959853 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ef930-5214-4f9d-b5ec-adc098d0c9c8 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:57.970009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ef930-5214-4f9d-b5ec-adc098d0c9c8 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:57.973579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76aefe-6fd7-4681-ac91-69fb0b5391d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:57.974203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76aefe-6fd7-4681-ac91-69fb0b5391d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:57.974483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76aefe-6fd7-4681-ac91-69fb0b5391d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:57.974754 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76aefe-6fd7-4681-ac91-69fb0b5391d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:58.032129 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76aefe-6fd7-4681-ac91-69fb0b5391d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:58.033564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76aefe-6fd7-4681-ac91-69fb0b5391d2 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:58.043193 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1075/2162] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:08:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:09:02.528688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce75a2-f9c3-4ae1-a114-b2993a4494d0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:02.529152 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce75a2-f9c3-4ae1-a114-b2993a4494d0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:02.529250 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce75a2-f9c3-4ae1-a114-b2993a4494d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:02.549785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2cce75a2-f9c3-4ae1-a114-b2993a4494d0 None None] MFA Rules not processed for user `9c0f1bfeb1464f87a7f3ea4add940861`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:02.551031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2cce75a2-f9c3-4ae1-a114-b2993a4494d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:02.618981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1088/2163] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:02.953459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76aefe-6fd7-4681-ac91-69fb0b5391d2 service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:03.030303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76aefe-6fd7-4681-ac91-69fb0b5391d2 service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:03.036241 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568751e-919f-4f33-800d-39a2596e7d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:03.037152 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568751e-919f-4f33-800d-39a2596e7d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:03.038568 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568751e-919f-4f33-800d-39a2596e7d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:03.038568 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568751e-919f-4f33-800d-39a2596e7d10 admin admin] PATH_INFO: `/v3/users/33e2f1bc844b4a259ab69b629888b244` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:03.039983 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568751e-919f-4f33-800d-39a2596e7d10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33e2f1bc844b4a259ab69b629888b244)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:03.042400 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568751e-919f-4f33-800d-39a2596e7d10 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:03.091712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:03.101738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c568751e-919f-4f33-800d-39a2596e7d10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33e2f1bc844b4a259ab69b629888b244'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:03.111710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c568751e-919f-4f33-800d-39a2596e7d10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33e2f1bc844b4a259ab69b629888b244'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:03.126260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c568751e-919f-4f33-800d-39a2596e7d10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33e2f1bc844b4a259ab69b629888b244'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:03.136358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c568751e-919f-4f33-800d-39a2596e7d10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33e2f1bc844b4a259ab69b629888b244'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:03.137216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c568751e-919f-4f33-800d-39a2596e7d10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33e2f1bc844b4a259ab69b629888b244'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:03.144856 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1076/2164] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:09:02 2022] DELETE /identity/v3/users/33e2f1bc844b4a259ab69b629888b244 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:03.151077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8f877-a125-4903-b6fc-ac598de1bd11 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:03.151400 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8f877-a125-4903-b6fc-ac598de1bd11 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:03.151807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8f877-a125-4903-b6fc-ac598de1bd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:03.172917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-bad8f877-a125-4903-b6fc-ac598de1bd11 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:03.174153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bad8f877-a125-4903-b6fc-ac598de1bd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:03.245844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1089/2165] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:03.256649 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568751e-919f-4f33-800d-39a2596e7d10 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:03.258938 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-c568751e-919f-4f33-800d-39a2596e7d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:03.344652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568751e-919f-4f33-800d-39a2596e7d10 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:03.348940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1a91e-b489-48fc-90b7-7776e5f79a6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:03.349659 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1a91e-b489-48fc-90b7-7776e5f79a6e service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:03.350171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1a91e-b489-48fc-90b7-7776e5f79a6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:03.350578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1a91e-b489-48fc-90b7-7776e5f79a6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:03.361914 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1a91e-b489-48fc-90b7-7776e5f79a6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:03.364711 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1a91e-b489-48fc-90b7-7776e5f79a6e service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:03.376371 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1077/2166] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:09:03.693414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8f877-a125-4903-b6fc-ac598de1bd11 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:03.695047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bad8f877-a125-4903-b6fc-ac598de1bd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:03.708040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8f877-a125-4903-b6fc-ac598de1bd11 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:03.714458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c5961-1238-41da-8bf0-8833dc287807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:03.715290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c5961-1238-41da-8bf0-8833dc287807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:03.715568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c5961-1238-41da-8bf0-8833dc287807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:03.715872 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c5961-1238-41da-8bf0-8833dc287807 admin admin] PATH_INFO: `/v3/projects/c0474b10d5104ac7a38196e80bfc2dfb` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:03.717020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c5961-1238-41da-8bf0-8833dc287807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0474b10d5104ac7a38196e80bfc2dfb)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:03.719135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c5961-1238-41da-8bf0-8833dc287807 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:03.722695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:03.773921 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d13c5961-1238-41da-8bf0-8833dc287807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0474b10d5104ac7a38196e80bfc2dfb', 'request_id': 'req-d13c5961-1238-41da-8bf0-8833dc287807'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:03.781394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d13c5961-1238-41da-8bf0-8833dc287807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0474b10d5104ac7a38196e80bfc2dfb', 'request_id': 'req-d13c5961-1238-41da-8bf0-8833dc287807'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:03.787040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d13c5961-1238-41da-8bf0-8833dc287807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0474b10d5104ac7a38196e80bfc2dfb', 'request_id': 'req-d13c5961-1238-41da-8bf0-8833dc287807'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:03.798753 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d13c5961-1238-41da-8bf0-8833dc287807 admin admin] The token cache is being invalidate because project c0474b10d5104ac7a38196e80bfc2dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:09:03.799046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d13c5961-1238-41da-8bf0-8833dc287807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:03.799689 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d13c5961-1238-41da-8bf0-8833dc287807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:03.801187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1090/2167] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:09:03 2022] DELETE /identity/v3/projects/c0474b10d5104ac7a38196e80bfc2dfb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:03.805264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1a91e-b489-48fc-90b7-7776e5f79a6e service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:03.814865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1a91e-b489-48fc-90b7-7776e5f79a6e service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:03.871975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4a43f-3579-4499-9af7-fdb198b202d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:03.872606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4a43f-3579-4499-9af7-fdb198b202d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:03.872887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4a43f-3579-4499-9af7-fdb198b202d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:03.873159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4a43f-3579-4499-9af7-fdb198b202d3 admin admin] PATH_INFO: `/v3/users/25f4e995dea147cb87f1e7fd832be438` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:03.874321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4a43f-3579-4499-9af7-fdb198b202d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25f4e995dea147cb87f1e7fd832be438)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:03.876270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4a43f-3579-4499-9af7-fdb198b202d3 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:03.927975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-62f4a43f-3579-4499-9af7-fdb198b202d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25f4e995dea147cb87f1e7fd832be438'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:03.941058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-62f4a43f-3579-4499-9af7-fdb198b202d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25f4e995dea147cb87f1e7fd832be438'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:03.947559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:03.957059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-62f4a43f-3579-4499-9af7-fdb198b202d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25f4e995dea147cb87f1e7fd832be438'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:03.965116 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-62f4a43f-3579-4499-9af7-fdb198b202d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25f4e995dea147cb87f1e7fd832be438'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:03.965853 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-62f4a43f-3579-4499-9af7-fdb198b202d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25f4e995dea147cb87f1e7fd832be438'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:03.971775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1078/2168] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:09:03 2022] DELETE /identity/v3/users/25f4e995dea147cb87f1e7fd832be438 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:04.249143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c5961-1238-41da-8bf0-8833dc287807 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:04.251439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d13c5961-1238-41da-8bf0-8833dc287807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:04.270035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c5961-1238-41da-8bf0-8833dc287807 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:04.327901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02647cad-7ab9-4148-a766-e9fa50701c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:04.328459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02647cad-7ab9-4148-a766-e9fa50701c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:04.328734 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02647cad-7ab9-4148-a766-e9fa50701c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:04.329010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02647cad-7ab9-4148-a766-e9fa50701c18 admin admin] PATH_INFO: `/v3/projects/4c723ba5d5bd42e28cb9e968dc1d3cb4` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:04.330151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02647cad-7ab9-4148-a766-e9fa50701c18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c723ba5d5bd42e28cb9e968dc1d3cb4)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:04.332086 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02647cad-7ab9-4148-a766-e9fa50701c18 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:04.378041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-02647cad-7ab9-4148-a766-e9fa50701c18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c723ba5d5bd42e28cb9e968dc1d3cb4', 'request_id': 'req-02647cad-7ab9-4148-a766-e9fa50701c18'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:04.385235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-02647cad-7ab9-4148-a766-e9fa50701c18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c723ba5d5bd42e28cb9e968dc1d3cb4', 'request_id': 'req-02647cad-7ab9-4148-a766-e9fa50701c18'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:04.391006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-02647cad-7ab9-4148-a766-e9fa50701c18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c723ba5d5bd42e28cb9e968dc1d3cb4', 'request_id': 'req-02647cad-7ab9-4148-a766-e9fa50701c18'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:04.405044 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-02647cad-7ab9-4148-a766-e9fa50701c18 admin admin] The token cache is being invalidate because project 4c723ba5d5bd42e28cb9e968dc1d3cb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:09:04.405334 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-02647cad-7ab9-4148-a766-e9fa50701c18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:04.405971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-02647cad-7ab9-4148-a766-e9fa50701c18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:04.407540 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1091/2169] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:09:04 2022] DELETE /identity/v3/projects/4c723ba5d5bd42e28cb9e968dc1d3cb4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:04.412289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971624f-2b9f-4c75-aeac-6dd3affe8aac None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:04.412793 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971624f-2b9f-4c75-aeac-6dd3affe8aac None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:04.413649 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971624f-2b9f-4c75-aeac-6dd3affe8aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:04.434003 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d971624f-2b9f-4c75-aeac-6dd3affe8aac None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:04.435243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d971624f-2b9f-4c75-aeac-6dd3affe8aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:04.505475 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1079/2170] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:04.507838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a96bf6-f0f8-4463-8a5d-37e066a92fcc None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:04.508081 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a96bf6-f0f8-4463-8a5d-37e066a92fcc None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:04.508349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a96bf6-f0f8-4463-8a5d-37e066a92fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:04.529546 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-55a96bf6-f0f8-4463-8a5d-37e066a92fcc None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:04.530768 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-55a96bf6-f0f8-4463-8a5d-37e066a92fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:04.539009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1092/2171] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:04.543218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971624f-2b9f-4c75-aeac-6dd3affe8aac None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:04.553025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971624f-2b9f-4c75-aeac-6dd3affe8aac None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:04.558331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fdcdf-58ba-4dc7-a158-f0c489a8c416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:04.558926 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fdcdf-58ba-4dc7-a158-f0c489a8c416 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:04.559207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fdcdf-58ba-4dc7-a158-f0c489a8c416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:04.559482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fdcdf-58ba-4dc7-a158-f0c489a8c416 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:04.560245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fdcdf-58ba-4dc7-a158-f0c489a8c416 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:09:04.560555 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fdcdf-58ba-4dc7-a158-f0c489a8c416 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:04.562241 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fdcdf-58ba-4dc7-a158-f0c489a8c416 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:04.574140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1080/2172] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:09:04.578657 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a96bf6-f0f8-4463-8a5d-37e066a92fcc None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:04.587800 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a96bf6-f0f8-4463-8a5d-37e066a92fcc None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:04.592442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bfb46-c33b-4a34-9395-42c887256ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:04.593027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bfb46-c33b-4a34-9395-42c887256ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:04.593299 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bfb46-c33b-4a34-9395-42c887256ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:04.593563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bfb46-c33b-4a34-9395-42c887256ff1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:04.594313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41bfb46-c33b-4a34-9395-42c887256ff1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:04.596179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41bfb46-c33b-4a34-9395-42c887256ff1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:04.611078 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:04.625869 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1093/2173] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:09:04 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:09:04.629948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fdcdf-58ba-4dc7-a158-f0c489a8c416 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:04.639923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fdcdf-58ba-4dc7-a158-f0c489a8c416 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:04.696579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56434af-0d1b-45ae-9f60-c32314f91cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:04.697181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56434af-0d1b-45ae-9f60-c32314f91cdf admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:04.697453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56434af-0d1b-45ae-9f60-c32314f91cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:04.697748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56434af-0d1b-45ae-9f60-c32314f91cdf admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:04.698400 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56434af-0d1b-45ae-9f60-c32314f91cdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:04.700307 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56434af-0d1b-45ae-9f60-c32314f91cdf admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:04.722287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1081/2174] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:09:04 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:09:04.726889 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bfb46-c33b-4a34-9395-42c887256ff1 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:04.737093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bfb46-c33b-4a34-9395-42c887256ff1 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:04.742341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f8270-c3b2-4ae9-adf0-985c84a79654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:04.742929 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f8270-c3b2-4ae9-adf0-985c84a79654 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:04.743200 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f8270-c3b2-4ae9-adf0-985c84a79654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:04.743463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f8270-c3b2-4ae9-adf0-985c84a79654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:04.744090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f8270-c3b2-4ae9-adf0-985c84a79654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:04.745921 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f8270-c3b2-4ae9-adf0-985c84a79654 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:04.755701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1094/2175] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:09:04.759536 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56434af-0d1b-45ae-9f60-c32314f91cdf admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:04.769445 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56434af-0d1b-45ae-9f60-c32314f91cdf admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:04.774442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c773e8-c227-4757-a8fe-7d8b246837fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:04.775944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c773e8-c227-4757-a8fe-7d8b246837fa admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:04.776236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c773e8-c227-4757-a8fe-7d8b246837fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:04.776540 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c773e8-c227-4757-a8fe-7d8b246837fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:04.777211 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c773e8-c227-4757-a8fe-7d8b246837fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:04.779698 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c773e8-c227-4757-a8fe-7d8b246837fa admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:04.791559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1082/2176] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:04.795761 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f8270-c3b2-4ae9-adf0-985c84a79654 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:04.806297 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f8270-c3b2-4ae9-adf0-985c84a79654 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:04.810915 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffb3d3-b69e-43d2-82d5-7a6a957a15c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:04.811558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffb3d3-b69e-43d2-82d5-7a6a957a15c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:04.811834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffb3d3-b69e-43d2-82d5-7a6a957a15c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:04.812101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffb3d3-b69e-43d2-82d5-7a6a957a15c5 admin admin] PATH_INFO: `/v3/projects/b014f4d058de4b4da33595cd51daa11b/users/ddcd02c96f6a4b149c50ef29eacd92c5/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:04.826537 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffb3d3-b69e-43d2-82d5-7a6a957a15c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b014f4d058de4b4da33595cd51daa11b, user_id=ddcd02c96f6a4b149c50ef29eacd92c5, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:04.829249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffb3d3-b69e-43d2-82d5-7a6a957a15c5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:04.843999 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1095/2177] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:09:04 2022] PUT /identity/v3/projects/b014f4d058de4b4da33595cd51daa11b/users/ddcd02c96f6a4b149c50ef29eacd92c5/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:09:04.854964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2257305-2311-474c-9c3c-b9da976aab77 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:04.855245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2257305-2311-474c-9c3c-b9da976aab77 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:04.855515 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2257305-2311-474c-9c3c-b9da976aab77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:04.882900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-b2257305-2311-474c-9c3c-b9da976aab77 None None] MFA Rules not processed for user `ddcd02c96f6a4b149c50ef29eacd92c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:04.884111 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b2257305-2311-474c-9c3c-b9da976aab77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:04.918295 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:04.951599 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1083/2178] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:04.962815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffb3d3-b69e-43d2-82d5-7a6a957a15c5 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:04.964513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-26ffb3d3-b69e-43d2-82d5-7a6a957a15c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:05.023638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffb3d3-b69e-43d2-82d5-7a6a957a15c5 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:05.027434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993817a-e0b5-4204-adf7-2e179fc32512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:05.028680 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993817a-e0b5-4204-adf7-2e179fc32512 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:05.028953 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993817a-e0b5-4204-adf7-2e179fc32512 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:05.029224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993817a-e0b5-4204-adf7-2e179fc32512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:05.037671 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993817a-e0b5-4204-adf7-2e179fc32512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:05.039173 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993817a-e0b5-4204-adf7-2e179fc32512 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:05.048056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1096/2179] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:09:09.274509 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dd323-7fd3-4101-b144-da42b03099cd None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:09.275078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dd323-7fd3-4101-b144-da42b03099cd None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:09.275348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dd323-7fd3-4101-b144-da42b03099cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:09.298846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-891dd323-7fd3-4101-b144-da42b03099cd None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:09.300179 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-891dd323-7fd3-4101-b144-da42b03099cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:09.359940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1084/2180] 10.222.0.62 () {58 vars in 1009 bytes} [Fri Sep 9 12:09:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:09:09.407956 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993817a-e0b5-4204-adf7-2e179fc32512 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:09.409497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0993817a-e0b5-4204-adf7-2e179fc32512 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:09.482701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993817a-e0b5-4204-adf7-2e179fc32512 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:09.488162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5d808-190b-4cd5-a8c6-a35035eef75e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:09.489001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5d808-190b-4cd5-a8c6-a35035eef75e service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:09.489138 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5d808-190b-4cd5-a8c6-a35035eef75e service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:09.489311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5d808-190b-4cd5-a8c6-a35035eef75e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:09.499214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5d808-190b-4cd5-a8c6-a35035eef75e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:09.500790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5d808-190b-4cd5-a8c6-a35035eef75e service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:09.510502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1097/2181] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:09:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:09:19.140042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dd323-7fd3-4101-b144-da42b03099cd None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:19.237157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dd323-7fd3-4101-b144-da42b03099cd None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:19.242312 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf59687-d792-4e1e-971f-6e6ebafb21f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:19.243182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf59687-d792-4e1e-971f-6e6ebafb21f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:19.243463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf59687-d792-4e1e-971f-6e6ebafb21f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:19.243992 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf59687-d792-4e1e-971f-6e6ebafb21f1 admin admin] PATH_INFO: `/v3/users/ddcd02c96f6a4b149c50ef29eacd92c5` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:19.245933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf59687-d792-4e1e-971f-6e6ebafb21f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddcd02c96f6a4b149c50ef29eacd92c5)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:19.249297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf59687-d792-4e1e-971f-6e6ebafb21f1 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:19.314672 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fdf59687-d792-4e1e-971f-6e6ebafb21f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddcd02c96f6a4b149c50ef29eacd92c5'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:19.326759 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fdf59687-d792-4e1e-971f-6e6ebafb21f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddcd02c96f6a4b149c50ef29eacd92c5'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:19.341203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fdf59687-d792-4e1e-971f-6e6ebafb21f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddcd02c96f6a4b149c50ef29eacd92c5'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:19.349145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fdf59687-d792-4e1e-971f-6e6ebafb21f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddcd02c96f6a4b149c50ef29eacd92c5'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:19.349868 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fdf59687-d792-4e1e-971f-6e6ebafb21f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddcd02c96f6a4b149c50ef29eacd92c5'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:19.351293 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:19.363822 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1085/2182] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:09:19 2022] DELETE /identity/v3/users/ddcd02c96f6a4b149c50ef29eacd92c5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:19.365639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d9871-5f3d-4671-a104-0c151dda3357 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:19.366989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d9871-5f3d-4671-a104-0c151dda3357 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:19.367551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d9871-5f3d-4671-a104-0c151dda3357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:19.396249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1d8d9871-5f3d-4671-a104-0c151dda3357 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:19.397490 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1d8d9871-5f3d-4671-a104-0c151dda3357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:19.477608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:19.497875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1098/2183] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:19.507477 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf59687-d792-4e1e-971f-6e6ebafb21f1 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:19.509053 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-fdf59687-d792-4e1e-971f-6e6ebafb21f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:19.585476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf59687-d792-4e1e-971f-6e6ebafb21f1 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:19.594538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae4ab9-09a6-45ac-95e1-5ec07d8c292b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:19.595541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae4ab9-09a6-45ac-95e1-5ec07d8c292b service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:19.595819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae4ab9-09a6-45ac-95e1-5ec07d8c292b service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:19.596133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae4ab9-09a6-45ac-95e1-5ec07d8c292b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:19.611050 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae4ab9-09a6-45ac-95e1-5ec07d8c292b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:19.611050 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae4ab9-09a6-45ac-95e1-5ec07d8c292b service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:19.622209 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1086/2184] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:09:20.303346 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d9871-5f3d-4671-a104-0c151dda3357 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:20.305069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1d8d9871-5f3d-4671-a104-0c151dda3357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:20.323997 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d9871-5f3d-4671-a104-0c151dda3357 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:20.352331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9befe72-57f1-4248-a52b-2c8e0128139f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:20.355610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9befe72-57f1-4248-a52b-2c8e0128139f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:20.355897 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9befe72-57f1-4248-a52b-2c8e0128139f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:20.356178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9befe72-57f1-4248-a52b-2c8e0128139f admin admin] PATH_INFO: `/v3/projects/b014f4d058de4b4da33595cd51daa11b` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:20.358038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9befe72-57f1-4248-a52b-2c8e0128139f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b014f4d058de4b4da33595cd51daa11b)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:20.360022 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9befe72-57f1-4248-a52b-2c8e0128139f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:20.435010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f9befe72-57f1-4248-a52b-2c8e0128139f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b014f4d058de4b4da33595cd51daa11b', 'request_id': 'req-f9befe72-57f1-4248-a52b-2c8e0128139f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:20.449420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f9befe72-57f1-4248-a52b-2c8e0128139f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b014f4d058de4b4da33595cd51daa11b', 'request_id': 'req-f9befe72-57f1-4248-a52b-2c8e0128139f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:20.458047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f9befe72-57f1-4248-a52b-2c8e0128139f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b014f4d058de4b4da33595cd51daa11b', 'request_id': 'req-f9befe72-57f1-4248-a52b-2c8e0128139f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:20.471640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f9befe72-57f1-4248-a52b-2c8e0128139f admin admin] The token cache is being invalidate because project b014f4d058de4b4da33595cd51daa11b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:09:20.472188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f9befe72-57f1-4248-a52b-2c8e0128139f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:20.473047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f9befe72-57f1-4248-a52b-2c8e0128139f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:20.476176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1099/2185] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:09:20 2022] DELETE /identity/v3/projects/b014f4d058de4b4da33595cd51daa11b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:20.481678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e046e-c741-4812-88de-ee361e723351 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:20.482340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e046e-c741-4812-88de-ee361e723351 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:20.482340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e046e-c741-4812-88de-ee361e723351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:20.505515 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-5d8e046e-c741-4812-88de-ee361e723351 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:20.507847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-5d8e046e-c741-4812-88de-ee361e723351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:20.610082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1087/2186] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:20.619528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bf00d-55b0-49cd-8b2f-c817ce6a6d48 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:20.619817 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bf00d-55b0-49cd-8b2f-c817ce6a6d48 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:20.620560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bf00d-55b0-49cd-8b2f-c817ce6a6d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:20.647286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b34bf00d-55b0-49cd-8b2f-c817ce6a6d48 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:20.648934 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b34bf00d-55b0-49cd-8b2f-c817ce6a6d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:20.661426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1100/2187] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:09:20.666482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e046e-c741-4812-88de-ee361e723351 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:20.680325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e046e-c741-4812-88de-ee361e723351 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:20.687752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70423c82-8633-4cf4-a83e-8370b04ca4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:20.688465 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70423c82-8633-4cf4-a83e-8370b04ca4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:20.688804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70423c82-8633-4cf4-a83e-8370b04ca4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:20.689151 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70423c82-8633-4cf4-a83e-8370b04ca4a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:20.689876 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70423c82-8633-4cf4-a83e-8370b04ca4a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:09:20.690385 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70423c82-8633-4cf4-a83e-8370b04ca4a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:20.692072 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70423c82-8633-4cf4-a83e-8370b04ca4a8 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:20.713270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1088/2188] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:09:20.716103 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bf00d-55b0-49cd-8b2f-c817ce6a6d48 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:20.728643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bf00d-55b0-49cd-8b2f-c817ce6a6d48 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:20.736006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed915e-6a81-4624-b3de-ffc6dc07e71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:20.736758 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed915e-6a81-4624-b3de-ffc6dc07e71f admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:20.737041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed915e-6a81-4624-b3de-ffc6dc07e71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:20.737313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed915e-6a81-4624-b3de-ffc6dc07e71f admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:20.737987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed915e-6a81-4624-b3de-ffc6dc07e71f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:20.739978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed915e-6a81-4624-b3de-ffc6dc07e71f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:20.776998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1101/2189] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:09:20 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:09:20.783718 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70423c82-8633-4cf4-a83e-8370b04ca4a8 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:20.797135 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70423c82-8633-4cf4-a83e-8370b04ca4a8 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:20.898357 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242295cc-bcbc-4bbb-890e-e7148d234dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:20.898979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242295cc-bcbc-4bbb-890e-e7148d234dfb admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:20.899260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242295cc-bcbc-4bbb-890e-e7148d234dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:20.899533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242295cc-bcbc-4bbb-890e-e7148d234dfb admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:20.900993 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242295cc-bcbc-4bbb-890e-e7148d234dfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:20.902956 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242295cc-bcbc-4bbb-890e-e7148d234dfb admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:20.953364 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1089/2190] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:09:20 2022] POST /identity/v3/users => generated 288 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:09:20.954792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed915e-6a81-4624-b3de-ffc6dc07e71f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:20.967637 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed915e-6a81-4624-b3de-ffc6dc07e71f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:20.994171 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442ba03-d1b6-4a74-a989-d26253569281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:20.994171 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442ba03-d1b6-4a74-a989-d26253569281 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:20.994171 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442ba03-d1b6-4a74-a989-d26253569281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:20.994171 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442ba03-d1b6-4a74-a989-d26253569281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:20.994766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442ba03-d1b6-4a74-a989-d26253569281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:20.994766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442ba03-d1b6-4a74-a989-d26253569281 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:21.005108 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1102/2191] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:21.011236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242295cc-bcbc-4bbb-890e-e7148d234dfb admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:21.022795 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242295cc-bcbc-4bbb-890e-e7148d234dfb admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:21.028728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6d2db-6ca0-4274-a427-bfe5bf54a17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:21.029363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6d2db-6ca0-4274-a427-bfe5bf54a17f admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:21.029646 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6d2db-6ca0-4274-a427-bfe5bf54a17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:21.029929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6d2db-6ca0-4274-a427-bfe5bf54a17f admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:21.030638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6d2db-6ca0-4274-a427-bfe5bf54a17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:21.042017 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6d2db-6ca0-4274-a427-bfe5bf54a17f admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:21.053546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1090/2192] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:21.055129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442ba03-d1b6-4a74-a989-d26253569281 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:21.066481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442ba03-d1b6-4a74-a989-d26253569281 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:21.083195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512f6aa-e187-4525-9e86-591633778d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:21.085008 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512f6aa-e187-4525-9e86-591633778d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:21.085008 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512f6aa-e187-4525-9e86-591633778d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:21.085008 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512f6aa-e187-4525-9e86-591633778d98 admin admin] PATH_INFO: `/v3/projects/a3b56272b6d2450487d0216e1741e6a1/users/3c7b26adfeff486f9012b0154d33a778/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:21.113256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6d2db-6ca0-4274-a427-bfe5bf54a17f admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:21.115001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-59e6d2db-6ca0-4274-a427-bfe5bf54a17f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:21.121324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512f6aa-e187-4525-9e86-591633778d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3b56272b6d2450487d0216e1741e6a1, user_id=3c7b26adfeff486f9012b0154d33a778, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:21.124611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512f6aa-e187-4525-9e86-591633778d98 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:21.127480 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:21.148527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1103/2193] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:09:21 2022] PUT /identity/v3/projects/a3b56272b6d2450487d0216e1741e6a1/users/3c7b26adfeff486f9012b0154d33a778/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:21.164792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0621c-a10d-4a0d-9a3b-e5fe1b0426f6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:21.165440 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0621c-a10d-4a0d-9a3b-e5fe1b0426f6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:21.165927 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0621c-a10d-4a0d-9a3b-e5fe1b0426f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:21.223617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ede0621c-a10d-4a0d-9a3b-e5fe1b0426f6 None None] MFA Rules not processed for user `3c7b26adfeff486f9012b0154d33a778`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:21.226245 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ede0621c-a10d-4a0d-9a3b-e5fe1b0426f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:21.270945 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6d2db-6ca0-4274-a427-bfe5bf54a17f admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:21.309045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:21.313294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1104/2194] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:21.317739 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0621c-a10d-4a0d-9a3b-e5fe1b0426f6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:21.382602 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc50296-856b-4f89-a6e3-580134d64ac5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:21.387454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc50296-856b-4f89-a6e3-580134d64ac5 service placement] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:21.388279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc50296-856b-4f89-a6e3-580134d64ac5 service placement] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:21.389065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc50296-856b-4f89-a6e3-580134d64ac5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:21.392939 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-bdc50296-856b-4f89-a6e3-580134d64ac5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:21.421897 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc50296-856b-4f89-a6e3-580134d64ac5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:21.423647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc50296-856b-4f89-a6e3-580134d64ac5 service placement] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:21.433424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0621c-a10d-4a0d-9a3b-e5fe1b0426f6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:21.438891 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eff13b-3995-48cb-8ce8-c228c9468567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:21.439544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eff13b-3995-48cb-8ce8-c228c9468567 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:21.439973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eff13b-3995-48cb-8ce8-c228c9468567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:21.440325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eff13b-3995-48cb-8ce8-c228c9468567 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:21.441047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eff13b-3995-48cb-8ce8-c228c9468567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:21.443344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eff13b-3995-48cb-8ce8-c228c9468567 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:21.448879 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1091/2195] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:09:21.480241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1105/2196] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:09:21 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:09:21.484137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc50296-856b-4f89-a6e3-580134d64ac5 service placement] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:21.498020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc50296-856b-4f89-a6e3-580134d64ac5 service placement] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:21.573799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11041e8a-1170-4a88-8f00-75a37184b742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:21.574567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11041e8a-1170-4a88-8f00-75a37184b742 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:21.574917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11041e8a-1170-4a88-8f00-75a37184b742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:21.575285 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11041e8a-1170-4a88-8f00-75a37184b742 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:21.576042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11041e8a-1170-4a88-8f00-75a37184b742 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:21.578521 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11041e8a-1170-4a88-8f00-75a37184b742 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:21.609815 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1092/2197] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:09:21 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:09:21.611525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eff13b-3995-48cb-8ce8-c228c9468567 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:21.625740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eff13b-3995-48cb-8ce8-c228c9468567 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:21.631168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b4afb-9cce-43d1-a1b7-dbf53160d0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:21.631856 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b4afb-9cce-43d1-a1b7-dbf53160d0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:21.632218 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b4afb-9cce-43d1-a1b7-dbf53160d0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:21.632699 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b4afb-9cce-43d1-a1b7-dbf53160d0f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:21.633420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b4afb-9cce-43d1-a1b7-dbf53160d0f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:21.635561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b4afb-9cce-43d1-a1b7-dbf53160d0f8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:21.650899 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1106/2198] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:21.652559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11041e8a-1170-4a88-8f00-75a37184b742 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:21.664493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11041e8a-1170-4a88-8f00-75a37184b742 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:21.670537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b276f0-d603-4dd4-86b6-a0e44fd38068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:21.671222 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b276f0-d603-4dd4-86b6-a0e44fd38068 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:21.671484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b276f0-d603-4dd4-86b6-a0e44fd38068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:21.672095 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b276f0-d603-4dd4-86b6-a0e44fd38068 admin admin] PATH_INFO: `/v3/projects/c84a599969c2494fb88ae362afaeb689/users/c64924f839004daba97d2fef0c707832/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:21.690541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b276f0-d603-4dd4-86b6-a0e44fd38068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84a599969c2494fb88ae362afaeb689, user_id=c64924f839004daba97d2fef0c707832, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:21.693967 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b276f0-d603-4dd4-86b6-a0e44fd38068 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:21.711892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1093/2199] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:09:21 2022] PUT /identity/v3/projects/c84a599969c2494fb88ae362afaeb689/users/c64924f839004daba97d2fef0c707832/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:21.716182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b4afb-9cce-43d1-a1b7-dbf53160d0f8 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:21.730076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b4afb-9cce-43d1-a1b7-dbf53160d0f8 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:21.839671 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fd8ad-2d4b-419e-8e66-eab763edb787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:21.841366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fd8ad-2d4b-419e-8e66-eab763edb787 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:21.841985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fd8ad-2d4b-419e-8e66-eab763edb787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:21.842492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fd8ad-2d4b-419e-8e66-eab763edb787 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:21.843501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fd8ad-2d4b-419e-8e66-eab763edb787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:21.846453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fd8ad-2d4b-419e-8e66-eab763edb787 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:21.864320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1107/2200] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:21.868339 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b276f0-d603-4dd4-86b6-a0e44fd38068 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:21.884305 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:21.897499 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b276f0-d603-4dd4-86b6-a0e44fd38068 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:21.903583 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105fb9f-eec1-4673-aae7-941ded9dbc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:21.904361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105fb9f-eec1-4673-aae7-941ded9dbc0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:21.904824 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105fb9f-eec1-4673-aae7-941ded9dbc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:21.905159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105fb9f-eec1-4673-aae7-941ded9dbc0f admin admin] PATH_INFO: `/v3/projects/c84a599969c2494fb88ae362afaeb689/users/c64924f839004daba97d2fef0c707832/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:21.907337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105fb9f-eec1-4673-aae7-941ded9dbc0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84a599969c2494fb88ae362afaeb689, user_id=c64924f839004daba97d2fef0c707832, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:21.910169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105fb9f-eec1-4673-aae7-941ded9dbc0f admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:21.923347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1094/2201] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:09:21 2022] PUT /identity/v3/projects/c84a599969c2494fb88ae362afaeb689/users/c64924f839004daba97d2fef0c707832/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:21.935495 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd4239-cee9-4ad1-8f3c-7dde695329f0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:21.935881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd4239-cee9-4ad1-8f3c-7dde695329f0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:21.936249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd4239-cee9-4ad1-8f3c-7dde695329f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:21.981048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-07fd4239-cee9-4ad1-8f3c-7dde695329f0 None None] MFA Rules not processed for user `c64924f839004daba97d2fef0c707832`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:21.982909 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-07fd4239-cee9-4ad1-8f3c-7dde695329f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:22.057186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:22.067401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1108/2202] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:22.080081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105fb9f-eec1-4673-aae7-941ded9dbc0f admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:22.082223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-1105fb9f-eec1-4673-aae7-941ded9dbc0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:22.149645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105fb9f-eec1-4673-aae7-941ded9dbc0f admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:22.155962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2760ef-293e-4259-80a2-d9938974be2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:22.156844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2760ef-293e-4259-80a2-d9938974be2d service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:22.157249 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2760ef-293e-4259-80a2-d9938974be2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:22.157586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2760ef-293e-4259-80a2-d9938974be2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:22.238682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2760ef-293e-4259-80a2-d9938974be2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:22.240231 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2760ef-293e-4259-80a2-d9938974be2d service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:22.251897 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1095/2203] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:09:26.090064 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd4239-cee9-4ad1-8f3c-7dde695329f0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:26.153409 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd4239-cee9-4ad1-8f3c-7dde695329f0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:26.158547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f7ac1-b564-4c13-af5f-62159740d3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:26.159192 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f7ac1-b564-4c13-af5f-62159740d3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:26.159470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f7ac1-b564-4c13-af5f-62159740d3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:26.159898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f7ac1-b564-4c13-af5f-62159740d3d8 admin admin] PATH_INFO: `/v3/users/3c7b26adfeff486f9012b0154d33a778` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:26.161241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f7ac1-b564-4c13-af5f-62159740d3d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c7b26adfeff486f9012b0154d33a778)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:26.163241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f7ac1-b564-4c13-af5f-62159740d3d8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:26.240878 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7e1f7ac1-b564-4c13-af5f-62159740d3d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c7b26adfeff486f9012b0154d33a778'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:26.241494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7e1f7ac1-b564-4c13-af5f-62159740d3d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c7b26adfeff486f9012b0154d33a778'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:26.250848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7e1f7ac1-b564-4c13-af5f-62159740d3d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c7b26adfeff486f9012b0154d33a778'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:26.259337 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7e1f7ac1-b564-4c13-af5f-62159740d3d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c7b26adfeff486f9012b0154d33a778'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:26.272451 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7e1f7ac1-b564-4c13-af5f-62159740d3d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c7b26adfeff486f9012b0154d33a778'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:26.278821 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1109/2204] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:09:26 2022] DELETE /identity/v3/users/3c7b26adfeff486f9012b0154d33a778 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:09:26.283499 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed74435-a601-4598-a8f1-03c10426d8f2 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:26.284056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed74435-a601-4598-a8f1-03c10426d8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:26.284056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed74435-a601-4598-a8f1-03c10426d8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:26.303965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-eed74435-a601-4598-a8f1-03c10426d8f2 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:26.305289 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-eed74435-a601-4598-a8f1-03c10426d8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:26.369309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1096/2205] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:26.379262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f7ac1-b564-4c13-af5f-62159740d3d8 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:26.380707 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7e1f7ac1-b564-4c13-af5f-62159740d3d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:26.441870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f7ac1-b564-4c13-af5f-62159740d3d8 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:26.445695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22b100-be45-48a5-bcf4-4c88befa4a5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:26.446342 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22b100-be45-48a5-bcf4-4c88befa4a5a service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:26.446789 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22b100-be45-48a5-bcf4-4c88befa4a5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:26.447034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22b100-be45-48a5-bcf4-4c88befa4a5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:26.463619 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22b100-be45-48a5-bcf4-4c88befa4a5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:26.465096 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22b100-be45-48a5-bcf4-4c88befa4a5a service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:26.475635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1110/2206] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:09:26.775964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed74435-a601-4598-a8f1-03c10426d8f2 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:26.778262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-eed74435-a601-4598-a8f1-03c10426d8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:26.796807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed74435-a601-4598-a8f1-03c10426d8f2 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:26.803132 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033452d-472d-49fd-a1de-e9adcd0a1796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:26.804259 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033452d-472d-49fd-a1de-e9adcd0a1796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:26.805026 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033452d-472d-49fd-a1de-e9adcd0a1796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:26.805262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033452d-472d-49fd-a1de-e9adcd0a1796 admin admin] PATH_INFO: `/v3/projects/a3b56272b6d2450487d0216e1741e6a1` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:26.807613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033452d-472d-49fd-a1de-e9adcd0a1796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3b56272b6d2450487d0216e1741e6a1)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:26.812267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033452d-472d-49fd-a1de-e9adcd0a1796 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:26.858587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:26.866243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4033452d-472d-49fd-a1de-e9adcd0a1796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3b56272b6d2450487d0216e1741e6a1', 'request_id': 'req-4033452d-472d-49fd-a1de-e9adcd0a1796'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:26.876052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4033452d-472d-49fd-a1de-e9adcd0a1796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3b56272b6d2450487d0216e1741e6a1', 'request_id': 'req-4033452d-472d-49fd-a1de-e9adcd0a1796'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:26.884547 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4033452d-472d-49fd-a1de-e9adcd0a1796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3b56272b6d2450487d0216e1741e6a1', 'request_id': 'req-4033452d-472d-49fd-a1de-e9adcd0a1796'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:26.899754 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4033452d-472d-49fd-a1de-e9adcd0a1796 admin admin] The token cache is being invalidate because project a3b56272b6d2450487d0216e1741e6a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:09:26.900094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4033452d-472d-49fd-a1de-e9adcd0a1796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:26.900790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4033452d-472d-49fd-a1de-e9adcd0a1796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:26.903136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1097/2207] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:09:26 2022] DELETE /identity/v3/projects/a3b56272b6d2450487d0216e1741e6a1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:26.908202 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22b100-be45-48a5-bcf4-4c88befa4a5a service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:26.918209 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22b100-be45-48a5-bcf4-4c88befa4a5a service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:26.931192 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:26.987076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278791a0-75d5-4694-8882-da3f063f17a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:26.988083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278791a0-75d5-4694-8882-da3f063f17a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:26.988436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278791a0-75d5-4694-8882-da3f063f17a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:26.988714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278791a0-75d5-4694-8882-da3f063f17a1 admin admin] PATH_INFO: `/v3/users/c64924f839004daba97d2fef0c707832` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:26.989974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278791a0-75d5-4694-8882-da3f063f17a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c64924f839004daba97d2fef0c707832)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:26.992008 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278791a0-75d5-4694-8882-da3f063f17a1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:27.049074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-278791a0-75d5-4694-8882-da3f063f17a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c64924f839004daba97d2fef0c707832'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:27.049976 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-278791a0-75d5-4694-8882-da3f063f17a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c64924f839004daba97d2fef0c707832'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:27.060028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-278791a0-75d5-4694-8882-da3f063f17a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c64924f839004daba97d2fef0c707832'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:27.069093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-278791a0-75d5-4694-8882-da3f063f17a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c64924f839004daba97d2fef0c707832'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:27.085934 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-278791a0-75d5-4694-8882-da3f063f17a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c64924f839004daba97d2fef0c707832'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:27.101196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1111/2208] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:09:26 2022] DELETE /identity/v3/users/c64924f839004daba97d2fef0c707832 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:27.158669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041ebaf-c332-410f-b7bf-81a36f3f5c4f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:27.158978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041ebaf-c332-410f-b7bf-81a36f3f5c4f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:27.159258 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041ebaf-c332-410f-b7bf-81a36f3f5c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:27.182356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-7041ebaf-c332-410f-b7bf-81a36f3f5c4f None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:27.183955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7041ebaf-c332-410f-b7bf-81a36f3f5c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:27.256062 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1098/2209] 10.222.0.62 () {58 vars in 967 bytes} [Fri Sep 9 12:09:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:27.275396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278791a0-75d5-4694-8882-da3f063f17a1 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:27.277306 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-278791a0-75d5-4694-8882-da3f063f17a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:27.367116 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278791a0-75d5-4694-8882-da3f063f17a1 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:27.373636 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4e40b-dd57-4a39-be37-cb5a83bec5c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:27.374303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4e40b-dd57-4a39-be37-cb5a83bec5c5 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:27.374579 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4e40b-dd57-4a39-be37-cb5a83bec5c5 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:27.375333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4e40b-dd57-4a39-be37-cb5a83bec5c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:27.385716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4e40b-dd57-4a39-be37-cb5a83bec5c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:27.387701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4e40b-dd57-4a39-be37-cb5a83bec5c5 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:27.397033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1112/2210] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:09:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:09:27.417631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041ebaf-c332-410f-b7bf-81a36f3f5c4f None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:27.419516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7041ebaf-c332-410f-b7bf-81a36f3f5c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:27.549015 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041ebaf-c332-410f-b7bf-81a36f3f5c4f None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:27.554580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a4515-500f-4315-bf59-36cf18f6b101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:27.555370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a4515-500f-4315-bf59-36cf18f6b101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:27.555515 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a4515-500f-4315-bf59-36cf18f6b101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:27.555786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a4515-500f-4315-bf59-36cf18f6b101 admin admin] PATH_INFO: `/v3/projects/c84a599969c2494fb88ae362afaeb689` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:27.557488 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a4515-500f-4315-bf59-36cf18f6b101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c84a599969c2494fb88ae362afaeb689)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:27.559631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a4515-500f-4315-bf59-36cf18f6b101 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:27.626194 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4a0a4515-500f-4315-bf59-36cf18f6b101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c84a599969c2494fb88ae362afaeb689', 'request_id': 'req-4a0a4515-500f-4315-bf59-36cf18f6b101'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:27.637826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4a0a4515-500f-4315-bf59-36cf18f6b101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c84a599969c2494fb88ae362afaeb689', 'request_id': 'req-4a0a4515-500f-4315-bf59-36cf18f6b101'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:27.648265 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4a0a4515-500f-4315-bf59-36cf18f6b101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c84a599969c2494fb88ae362afaeb689', 'request_id': 'req-4a0a4515-500f-4315-bf59-36cf18f6b101'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:27.664347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4a0a4515-500f-4315-bf59-36cf18f6b101 admin admin] The token cache is being invalidate because project c84a599969c2494fb88ae362afaeb689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:09:27.664651 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4a0a4515-500f-4315-bf59-36cf18f6b101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:27.665954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4a0a4515-500f-4315-bf59-36cf18f6b101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:27.667724 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1099/2211] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:09:27 2022] DELETE /identity/v3/projects/c84a599969c2494fb88ae362afaeb689 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:27.672483 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb8c70-18fa-49e1-b08c-056db2e84cc0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:27.672779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb8c70-18fa-49e1-b08c-056db2e84cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:27.673056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb8c70-18fa-49e1-b08c-056db2e84cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:27.698075 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f5eb8c70-18fa-49e1-b08c-056db2e84cc0 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:27.699518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f5eb8c70-18fa-49e1-b08c-056db2e84cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:27.725319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:27.786084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1113/2212] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:27.788272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe99a7-f48f-45e9-a91e-70b99fb46269 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:27.788946 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe99a7-f48f-45e9-a91e-70b99fb46269 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:27.789303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe99a7-f48f-45e9-a91e-70b99fb46269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:27.800942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:27.814842 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-9ebe99a7-f48f-45e9-a91e-70b99fb46269 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:27.816216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-9ebe99a7-f48f-45e9-a91e-70b99fb46269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:27.827615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1100/2213] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:27.833284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb8c70-18fa-49e1-b08c-056db2e84cc0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:27.843644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb8c70-18fa-49e1-b08c-056db2e84cc0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:27.851412 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd1222-64c7-4d45-9b88-09519c6c7790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:27.851412 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd1222-64c7-4d45-9b88-09519c6c7790 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:27.851845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd1222-64c7-4d45-9b88-09519c6c7790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:27.852153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd1222-64c7-4d45-9b88-09519c6c7790 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:27.853436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd1222-64c7-4d45-9b88-09519c6c7790 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:09:27.853816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd1222-64c7-4d45-9b88-09519c6c7790 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:27.855777 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd1222-64c7-4d45-9b88-09519c6c7790 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:27.871843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1114/2214] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:09:27.876011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe99a7-f48f-45e9-a91e-70b99fb46269 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:27.891622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe99a7-f48f-45e9-a91e-70b99fb46269 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:27.903750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440110aa-b5c0-4aee-a6dc-34c117750a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:27.903750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440110aa-b5c0-4aee-a6dc-34c117750a66 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:27.903750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440110aa-b5c0-4aee-a6dc-34c117750a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:27.903750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440110aa-b5c0-4aee-a6dc-34c117750a66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:27.904255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440110aa-b5c0-4aee-a6dc-34c117750a66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:27.905499 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440110aa-b5c0-4aee-a6dc-34c117750a66 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:27.963709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1101/2215] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:09:27 2022] POST /identity/v3/projects => generated 377 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:09:27.970491 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd1222-64c7-4d45-9b88-09519c6c7790 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:27.987265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd1222-64c7-4d45-9b88-09519c6c7790 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:28.069627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5b12c-223a-401a-ba13-a6e7a01cf36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:28.070687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5b12c-223a-401a-ba13-a6e7a01cf36d admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:28.071126 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5b12c-223a-401a-ba13-a6e7a01cf36d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:28.072119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5b12c-223a-401a-ba13-a6e7a01cf36d admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:28.073097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5b12c-223a-401a-ba13-a6e7a01cf36d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:28.076019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5b12c-223a-401a-ba13-a6e7a01cf36d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:28.101119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1115/2216] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:09:27 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:09:28.105893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440110aa-b5c0-4aee-a6dc-34c117750a66 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:28.118957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440110aa-b5c0-4aee-a6dc-34c117750a66 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:28.126823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa782da-71b6-49a8-9495-d4df06efbba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:28.128081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa782da-71b6-49a8-9495-d4df06efbba5 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:28.128382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa782da-71b6-49a8-9495-d4df06efbba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:28.128662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa782da-71b6-49a8-9495-d4df06efbba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:28.129369 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa782da-71b6-49a8-9495-d4df06efbba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:28.131199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa782da-71b6-49a8-9495-d4df06efbba5 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:28.148773 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1102/2217] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:28.150910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5b12c-223a-401a-ba13-a6e7a01cf36d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:28.162227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5b12c-223a-401a-ba13-a6e7a01cf36d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:28.170325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c49c34-fd72-4179-8c91-faf6b966531f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:28.170982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c49c34-fd72-4179-8c91-faf6b966531f admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:28.171313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c49c34-fd72-4179-8c91-faf6b966531f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:28.171625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c49c34-fd72-4179-8c91-faf6b966531f admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:28.172342 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c49c34-fd72-4179-8c91-faf6b966531f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:28.174735 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c49c34-fd72-4179-8c91-faf6b966531f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:28.187626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1116/2218] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:28.197194 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa782da-71b6-49a8-9495-d4df06efbba5 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:28.214612 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa782da-71b6-49a8-9495-d4df06efbba5 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:28.230464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f32bc-b640-438e-93f1-1c9b9e53cba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:28.231533 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f32bc-b640-438e-93f1-1c9b9e53cba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:28.232119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f32bc-b640-438e-93f1-1c9b9e53cba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:28.232458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f32bc-b640-438e-93f1-1c9b9e53cba0 admin admin] PATH_INFO: `/v3/projects/671ac82e302b48c4a32bb1e65dca4b61/users/8f7f35ed4dab44c6adf390da3649a0bd/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:28.254837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f32bc-b640-438e-93f1-1c9b9e53cba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=671ac82e302b48c4a32bb1e65dca4b61, user_id=8f7f35ed4dab44c6adf390da3649a0bd, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:28.257614 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f32bc-b640-438e-93f1-1c9b9e53cba0 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:28.278230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1103/2219] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:09:28 2022] PUT /identity/v3/projects/671ac82e302b48c4a32bb1e65dca4b61/users/8f7f35ed4dab44c6adf390da3649a0bd/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:28.288181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db650a-80d4-4b93-8fb9-b3e138aaee29 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:28.288526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db650a-80d4-4b93-8fb9-b3e138aaee29 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:28.289395 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db650a-80d4-4b93-8fb9-b3e138aaee29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:28.320338 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-61db650a-80d4-4b93-8fb9-b3e138aaee29 None None] MFA Rules not processed for user `8f7f35ed4dab44c6adf390da3649a0bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:28.321467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-61db650a-80d4-4b93-8fb9-b3e138aaee29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:28.392212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1117/2220] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:28.397113 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f32bc-b640-438e-93f1-1c9b9e53cba0 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:28.463884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f32bc-b640-438e-93f1-1c9b9e53cba0 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:28.468541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1713cd-98ef-4731-b113-0f87ecbc3572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:28.469196 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1713cd-98ef-4731-b113-0f87ecbc3572 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:28.469486 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1713cd-98ef-4731-b113-0f87ecbc3572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:28.469755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1713cd-98ef-4731-b113-0f87ecbc3572 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:28.470432 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1713cd-98ef-4731-b113-0f87ecbc3572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:28.472351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1713cd-98ef-4731-b113-0f87ecbc3572 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:28.501885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1104/2221] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:09:28 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:09:28.506063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db650a-80d4-4b93-8fb9-b3e138aaee29 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:28.517998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db650a-80d4-4b93-8fb9-b3e138aaee29 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:28.537541 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:28.580096 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849adca-7024-45a8-b129-010df6522af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:28.580625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849adca-7024-45a8-b129-010df6522af0 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:28.580895 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849adca-7024-45a8-b129-010df6522af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:28.581159 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849adca-7024-45a8-b129-010df6522af0 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:28.581821 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849adca-7024-45a8-b129-010df6522af0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:28.583758 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849adca-7024-45a8-b129-010df6522af0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:28.606141 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1118/2222] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:09:28 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:09:28.610149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1713cd-98ef-4731-b113-0f87ecbc3572 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:28.621219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1713cd-98ef-4731-b113-0f87ecbc3572 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:28.626862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307b3b6-6ba7-4510-bbc2-ee9704fb9da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:28.627713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307b3b6-6ba7-4510-bbc2-ee9704fb9da9 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:28.627914 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307b3b6-6ba7-4510-bbc2-ee9704fb9da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:28.628281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307b3b6-6ba7-4510-bbc2-ee9704fb9da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:28.629043 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307b3b6-6ba7-4510-bbc2-ee9704fb9da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:28.631187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307b3b6-6ba7-4510-bbc2-ee9704fb9da9 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:28.642428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1105/2223] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:28.646409 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849adca-7024-45a8-b129-010df6522af0 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:28.657335 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849adca-7024-45a8-b129-010df6522af0 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:28.662057 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67ce71-154e-43a1-9e22-19dd52212e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:28.662745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67ce71-154e-43a1-9e22-19dd52212e48 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:28.663023 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67ce71-154e-43a1-9e22-19dd52212e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:28.663787 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67ce71-154e-43a1-9e22-19dd52212e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:28.664066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67ce71-154e-43a1-9e22-19dd52212e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:28.665519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67ce71-154e-43a1-9e22-19dd52212e48 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:28.676604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1119/2224] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:28.680768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307b3b6-6ba7-4510-bbc2-ee9704fb9da9 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:28.692419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307b3b6-6ba7-4510-bbc2-ee9704fb9da9 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:28.697603 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9763b-0800-490e-be63-ea2b799a74dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:28.698302 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9763b-0800-490e-be63-ea2b799a74dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:28.698576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9763b-0800-490e-be63-ea2b799a74dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:28.698842 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9763b-0800-490e-be63-ea2b799a74dd admin admin] PATH_INFO: `/v3/projects/bbdbc7cb60ef4cfd94e127e650a4dcc0/users/97e42239729343f58bccef35a490842c/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:28.715524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a9763b-0800-490e-be63-ea2b799a74dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbdbc7cb60ef4cfd94e127e650a4dcc0, user_id=97e42239729343f58bccef35a490842c, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:28.718392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a9763b-0800-490e-be63-ea2b799a74dd admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:28.732668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1106/2225] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:09:28 2022] PUT /identity/v3/projects/bbdbc7cb60ef4cfd94e127e650a4dcc0/users/97e42239729343f58bccef35a490842c/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:28.744855 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1173041-620b-41c0-9c97-85e8f22aa8a7 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:28.745421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1173041-620b-41c0-9c97-85e8f22aa8a7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:28.745898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1173041-620b-41c0-9c97-85e8f22aa8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:28.776815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b1173041-620b-41c0-9c97-85e8f22aa8a7 None None] MFA Rules not processed for user `97e42239729343f58bccef35a490842c`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:28.778372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b1173041-620b-41c0-9c97-85e8f22aa8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:28.846259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1120/2226] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:28.850722 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9763b-0800-490e-be63-ea2b799a74dd admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:28.906802 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:28.919962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9763b-0800-490e-be63-ea2b799a74dd admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:28.925168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d3cfe-3df2-47b8-a2de-c328d1633aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:28.925773 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d3cfe-3df2-47b8-a2de-c328d1633aed admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:28.926048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d3cfe-3df2-47b8-a2de-c328d1633aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:28.926348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d3cfe-3df2-47b8-a2de-c328d1633aed admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:28.926991 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d3cfe-3df2-47b8-a2de-c328d1633aed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:28.928937 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d3cfe-3df2-47b8-a2de-c328d1633aed admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:28.958000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1107/2227] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:09:28 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:09:28.962069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1173041-620b-41c0-9c97-85e8f22aa8a7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:28.973396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1173041-620b-41c0-9c97-85e8f22aa8a7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:29.034519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537a5de-13c6-4108-b414-56af405103a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:29.035131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537a5de-13c6-4108-b414-56af405103a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:29.035403 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537a5de-13c6-4108-b414-56af405103a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:29.035668 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537a5de-13c6-4108-b414-56af405103a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:29.036302 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537a5de-13c6-4108-b414-56af405103a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:29.038242 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537a5de-13c6-4108-b414-56af405103a1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:29.064705 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1121/2228] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:09:28 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:09:29.065943 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d3cfe-3df2-47b8-a2de-c328d1633aed admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:29.077692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d3cfe-3df2-47b8-a2de-c328d1633aed admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:29.083317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb7589-78b7-483c-b8b0-a7417e3d31e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:29.084122 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb7589-78b7-483c-b8b0-a7417e3d31e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:29.084564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb7589-78b7-483c-b8b0-a7417e3d31e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:29.084916 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb7589-78b7-483c-b8b0-a7417e3d31e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:29.085830 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb7589-78b7-483c-b8b0-a7417e3d31e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:29.087615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb7589-78b7-483c-b8b0-a7417e3d31e5 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:29.099360 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1108/2229] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:29.103547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537a5de-13c6-4108-b414-56af405103a1 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:29.114693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537a5de-13c6-4108-b414-56af405103a1 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:29.120436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7090d3a-a0c9-466a-967c-a63d56579120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:29.121240 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7090d3a-a0c9-466a-967c-a63d56579120 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:29.121377 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7090d3a-a0c9-466a-967c-a63d56579120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:29.121646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7090d3a-a0c9-466a-967c-a63d56579120 admin admin] PATH_INFO: `/v3/projects/70f2600f735440b1adf201ae1f05ddc7/users/b2ea893959d74e88b10ba220a4ca6ae5/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:29.138912 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7090d3a-a0c9-466a-967c-a63d56579120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70f2600f735440b1adf201ae1f05ddc7, user_id=b2ea893959d74e88b10ba220a4ca6ae5, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:29.141584 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7090d3a-a0c9-466a-967c-a63d56579120 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:29.145039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:29.162383 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1122/2230] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:09:29 2022] PUT /identity/v3/projects/70f2600f735440b1adf201ae1f05ddc7/users/b2ea893959d74e88b10ba220a4ca6ae5/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:29.163859 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb7589-78b7-483c-b8b0-a7417e3d31e5 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:29.176791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb7589-78b7-483c-b8b0-a7417e3d31e5 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:29.243207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea1442-e280-4884-a853-9ee718c71321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:29.244397 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea1442-e280-4884-a853-9ee718c71321 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:29.244678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea1442-e280-4884-a853-9ee718c71321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:29.244975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea1442-e280-4884-a853-9ee718c71321 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:29.245921 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea1442-e280-4884-a853-9ee718c71321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:29.247590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea1442-e280-4884-a853-9ee718c71321 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:29.259918 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1109/2231] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:09:29.263988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7090d3a-a0c9-466a-967c-a63d56579120 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:29.275389 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7090d3a-a0c9-466a-967c-a63d56579120 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:29.280546 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc183f-fbaa-4c99-9964-7e61f91f6426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:29.281234 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc183f-fbaa-4c99-9964-7e61f91f6426 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:29.281542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc183f-fbaa-4c99-9964-7e61f91f6426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:29.281818 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc183f-fbaa-4c99-9964-7e61f91f6426 admin admin] PATH_INFO: `/v3/projects/70f2600f735440b1adf201ae1f05ddc7/users/b2ea893959d74e88b10ba220a4ca6ae5/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:29.283817 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc183f-fbaa-4c99-9964-7e61f91f6426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70f2600f735440b1adf201ae1f05ddc7, user_id=b2ea893959d74e88b10ba220a4ca6ae5, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:29.286697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc183f-fbaa-4c99-9964-7e61f91f6426 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:29.300203 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1123/2232] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:09:29 2022] PUT /identity/v3/projects/70f2600f735440b1adf201ae1f05ddc7/users/b2ea893959d74e88b10ba220a4ca6ae5/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:09:29.312132 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e5533-9180-45c1-8bc1-edf0a9b3be08 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:29.312465 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e5533-9180-45c1-8bc1-edf0a9b3be08 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:29.312830 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e5533-9180-45c1-8bc1-edf0a9b3be08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:29.347364 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-de6e5533-9180-45c1-8bc1-edf0a9b3be08 None None] MFA Rules not processed for user `b2ea893959d74e88b10ba220a4ca6ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:29.348902 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-de6e5533-9180-45c1-8bc1-edf0a9b3be08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:29.437975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1110/2233] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:29.450164 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc183f-fbaa-4c99-9964-7e61f91f6426 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:29.451925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-57dc183f-fbaa-4c99-9964-7e61f91f6426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:29.521709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc183f-fbaa-4c99-9964-7e61f91f6426 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:29.526405 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb6a79-fa63-4501-b5aa-85ecdd2abb18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:29.527713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb6a79-fa63-4501-b5aa-85ecdd2abb18 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:29.528195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb6a79-fa63-4501-b5aa-85ecdd2abb18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:29.528487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb6a79-fa63-4501-b5aa-85ecdd2abb18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:29.609974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb6a79-fa63-4501-b5aa-85ecdd2abb18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:29.611454 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb6a79-fa63-4501-b5aa-85ecdd2abb18 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:29.624813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1124/2234] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:09:30.112552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e5533-9180-45c1-8bc1-edf0a9b3be08 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:30.114069 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-de6e5533-9180-45c1-8bc1-edf0a9b3be08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:30.168860 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:30.186741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e5533-9180-45c1-8bc1-edf0a9b3be08 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:30.191329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ff830-c99c-4970-8632-a937ee4295ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:30.191998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ff830-c99c-4970-8632-a937ee4295ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:30.192277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ff830-c99c-4970-8632-a937ee4295ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:30.192550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ff830-c99c-4970-8632-a937ee4295ec admin admin] PATH_INFO: `/v3/users/eef1afc3ece940248494501dd27e3920` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:30.193696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ff830-c99c-4970-8632-a937ee4295ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=eef1afc3ece940248494501dd27e3920)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:30.195681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ff830-c99c-4970-8632-a937ee4295ec admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:30.250414 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5f0ff830-c99c-4970-8632-a937ee4295ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eef1afc3ece940248494501dd27e3920'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:30.257793 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5f0ff830-c99c-4970-8632-a937ee4295ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eef1afc3ece940248494501dd27e3920'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:30.270503 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5f0ff830-c99c-4970-8632-a937ee4295ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eef1afc3ece940248494501dd27e3920'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:30.277650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5f0ff830-c99c-4970-8632-a937ee4295ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eef1afc3ece940248494501dd27e3920'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:30.278348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5f0ff830-c99c-4970-8632-a937ee4295ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eef1afc3ece940248494501dd27e3920'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:30.284133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1111/2235] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:09:30 2022] DELETE /identity/v3/users/eef1afc3ece940248494501dd27e3920 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:30.458201 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb6a79-fa63-4501-b5aa-85ecdd2abb18 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:30.471195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb6a79-fa63-4501-b5aa-85ecdd2abb18 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:30.533188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f13c89-5ade-4089-872b-eefd1e973a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:30.533823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f13c89-5ade-4089-872b-eefd1e973a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:30.534105 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f13c89-5ade-4089-872b-eefd1e973a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:30.534506 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f13c89-5ade-4089-872b-eefd1e973a2c admin admin] PATH_INFO: `/v3/projects/f85c5e478d5c4efca84cb570537ea15d` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:30.535638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f13c89-5ade-4089-872b-eefd1e973a2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f85c5e478d5c4efca84cb570537ea15d)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:30.537440 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f13c89-5ade-4089-872b-eefd1e973a2c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:30.578212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:30.586284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-88f13c89-5ade-4089-872b-eefd1e973a2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f85c5e478d5c4efca84cb570537ea15d', 'request_id': 'req-88f13c89-5ade-4089-872b-eefd1e973a2c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:30.595106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-88f13c89-5ade-4089-872b-eefd1e973a2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f85c5e478d5c4efca84cb570537ea15d', 'request_id': 'req-88f13c89-5ade-4089-872b-eefd1e973a2c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:30.600967 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-88f13c89-5ade-4089-872b-eefd1e973a2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f85c5e478d5c4efca84cb570537ea15d', 'request_id': 'req-88f13c89-5ade-4089-872b-eefd1e973a2c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:30.612814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-88f13c89-5ade-4089-872b-eefd1e973a2c admin admin] The token cache is being invalidate because project f85c5e478d5c4efca84cb570537ea15d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:09:30.613103 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-88f13c89-5ade-4089-872b-eefd1e973a2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:30.613743 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-88f13c89-5ade-4089-872b-eefd1e973a2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:30.615345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1125/2236] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:09:30 2022] DELETE /identity/v3/projects/f85c5e478d5c4efca84cb570537ea15d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:30.620660 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16682c16-99f1-40a8-9f0d-f9ed011db703 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:30.620946 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16682c16-99f1-40a8-9f0d-f9ed011db703 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:30.621218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16682c16-99f1-40a8-9f0d-f9ed011db703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:30.641637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-16682c16-99f1-40a8-9f0d-f9ed011db703 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:30.642862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-16682c16-99f1-40a8-9f0d-f9ed011db703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:30.701466 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1112/2237] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:30.709257 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fb362-b7c3-4b33-9062-a8f9ea3faee6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:30.709540 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fb362-b7c3-4b33-9062-a8f9ea3faee6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:30.709815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fb362-b7c3-4b33-9062-a8f9ea3faee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:30.730270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-843fb362-b7c3-4b33-9062-a8f9ea3faee6 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:30.731481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-843fb362-b7c3-4b33-9062-a8f9ea3faee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:30.739371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1126/2238] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:30.743560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16682c16-99f1-40a8-9f0d-f9ed011db703 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:30.753384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16682c16-99f1-40a8-9f0d-f9ed011db703 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:30.758022 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814516f-4971-4e25-b730-cded93930947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:30.758647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814516f-4971-4e25-b730-cded93930947 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:30.759410 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814516f-4971-4e25-b730-cded93930947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:30.759689 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814516f-4971-4e25-b730-cded93930947 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:30.760338 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814516f-4971-4e25-b730-cded93930947 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:09:30.760638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814516f-4971-4e25-b730-cded93930947 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:30.762360 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814516f-4971-4e25-b730-cded93930947 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:30.773888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1113/2239] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:09:30.778811 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fb362-b7c3-4b33-9062-a8f9ea3faee6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:30.788040 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fb362-b7c3-4b33-9062-a8f9ea3faee6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:30.792719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae1e34-b0a6-4633-b85f-1c4c680a5df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:30.793321 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae1e34-b0a6-4633-b85f-1c4c680a5df6 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:30.793600 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae1e34-b0a6-4633-b85f-1c4c680a5df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:30.793901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae1e34-b0a6-4633-b85f-1c4c680a5df6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:30.794586 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae1e34-b0a6-4633-b85f-1c4c680a5df6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:30.796468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae1e34-b0a6-4633-b85f-1c4c680a5df6 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:30.824142 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1127/2240] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:09:30 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:09:30.828596 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814516f-4971-4e25-b730-cded93930947 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:30.839262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814516f-4971-4e25-b730-cded93930947 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:30.887948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:30.901680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e8ecb-3f0a-4dab-bb39-f4687b04f8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:30.902350 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e8ecb-3f0a-4dab-bb39-f4687b04f8f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:30.902756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e8ecb-3f0a-4dab-bb39-f4687b04f8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:30.903201 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e8ecb-3f0a-4dab-bb39-f4687b04f8f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:30.903885 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e8ecb-3f0a-4dab-bb39-f4687b04f8f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:30.905960 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e8ecb-3f0a-4dab-bb39-f4687b04f8f8 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:30.928050 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1114/2241] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:09:30 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:09:30.932247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae1e34-b0a6-4633-b85f-1c4c680a5df6 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:30.942503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae1e34-b0a6-4633-b85f-1c4c680a5df6 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:30.947667 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18076f58-7f19-401a-be7a-f9728042fa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:30.948261 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18076f58-7f19-401a-be7a-f9728042fa40 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:30.948538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18076f58-7f19-401a-be7a-f9728042fa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:30.948811 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18076f58-7f19-401a-be7a-f9728042fa40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:30.949452 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18076f58-7f19-401a-be7a-f9728042fa40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:30.951069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18076f58-7f19-401a-be7a-f9728042fa40 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:30.962680 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1128/2242] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:30.967501 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e8ecb-3f0a-4dab-bb39-f4687b04f8f8 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:30.981744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e8ecb-3f0a-4dab-bb39-f4687b04f8f8 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:30.986852 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb2466-7ffb-46b2-b87b-53d0f4d82399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:30.987148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb2466-7ffb-46b2-b87b-53d0f4d82399 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:30.987426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb2466-7ffb-46b2-b87b-53d0f4d82399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:30.987699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb2466-7ffb-46b2-b87b-53d0f4d82399 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:30.988415 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb2466-7ffb-46b2-b87b-53d0f4d82399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:30.990215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb2466-7ffb-46b2-b87b-53d0f4d82399 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:31.004111 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1115/2243] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:31.008413 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18076f58-7f19-401a-be7a-f9728042fa40 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:31.020030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18076f58-7f19-401a-be7a-f9728042fa40 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:31.026769 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529549d-6d47-4369-ad5e-104437602acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:31.027976 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529549d-6d47-4369-ad5e-104437602acd admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:31.028492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529549d-6d47-4369-ad5e-104437602acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:31.028984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529549d-6d47-4369-ad5e-104437602acd admin admin] PATH_INFO: `/v3/projects/df92c18e38a3473eb221a063734ec863/users/67d0582228b043c28fbcdb2b60345f21/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:31.049021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529549d-6d47-4369-ad5e-104437602acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=df92c18e38a3473eb221a063734ec863, user_id=67d0582228b043c28fbcdb2b60345f21, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:31.052011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529549d-6d47-4369-ad5e-104437602acd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:31.072459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1129/2244] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:09:31 2022] PUT /identity/v3/projects/df92c18e38a3473eb221a063734ec863/users/67d0582228b043c28fbcdb2b60345f21/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:31.083753 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3c3ab-9f2c-4a7a-80eb-b1a2053455a3 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:31.084046 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3c3ab-9f2c-4a7a-80eb-b1a2053455a3 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:31.084321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3c3ab-9f2c-4a7a-80eb-b1a2053455a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:31.121190 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-faf3c3ab-9f2c-4a7a-80eb-b1a2053455a3 None None] MFA Rules not processed for user `67d0582228b043c28fbcdb2b60345f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:31.122462 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-faf3c3ab-9f2c-4a7a-80eb-b1a2053455a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:31.167992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529549d-6d47-4369-ad5e-104437602acd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:31.169696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5529549d-6d47-4369-ad5e-104437602acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:31.189893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1116/2245] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:31.201732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3c3ab-9f2c-4a7a-80eb-b1a2053455a3 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:31.203139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-faf3c3ab-9f2c-4a7a-80eb-b1a2053455a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:31.230430 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529549d-6d47-4369-ad5e-104437602acd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:31.234339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04decc67-53cf-4e9b-8435-d5088bb0bf50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:31.234962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04decc67-53cf-4e9b-8435-d5088bb0bf50 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:31.235232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04decc67-53cf-4e9b-8435-d5088bb0bf50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:31.235510 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04decc67-53cf-4e9b-8435-d5088bb0bf50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:31.237254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-04decc67-53cf-4e9b-8435-d5088bb0bf50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:31.261123 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3c3ab-9f2c-4a7a-80eb-b1a2053455a3 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:31.264737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40b7a0-64da-43c3-bfbb-ee97d6b1d2fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:31.265337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40b7a0-64da-43c3-bfbb-ee97d6b1d2fb service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:31.265613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40b7a0-64da-43c3-bfbb-ee97d6b1d2fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:31.265877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40b7a0-64da-43c3-bfbb-ee97d6b1d2fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:31.274337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40b7a0-64da-43c3-bfbb-ee97d6b1d2fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:31.275788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40b7a0-64da-43c3-bfbb-ee97d6b1d2fb service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:31.284875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1117/2246] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:09:31.298824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04decc67-53cf-4e9b-8435-d5088bb0bf50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:31.300248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04decc67-53cf-4e9b-8435-d5088bb0bf50 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:31.310012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1130/2247] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:09:31.889392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40b7a0-64da-43c3-bfbb-ee97d6b1d2fb service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:31.897959 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40b7a0-64da-43c3-bfbb-ee97d6b1d2fb service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:31.902208 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ebd0a-8c36-4456-b75f-ba27766bae67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:31.902795 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ebd0a-8c36-4456-b75f-ba27766bae67 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:31.903069 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ebd0a-8c36-4456-b75f-ba27766bae67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:31.903338 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ebd0a-8c36-4456-b75f-ba27766bae67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:31.905055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-776ebd0a-8c36-4456-b75f-ba27766bae67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:31.967374 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ebd0a-8c36-4456-b75f-ba27766bae67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:31.968799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ebd0a-8c36-4456-b75f-ba27766bae67 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:31.972727 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:31.980888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1118/2248] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:09:33.868463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04decc67-53cf-4e9b-8435-d5088bb0bf50 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:33.888181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:33.935917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04decc67-53cf-4e9b-8435-d5088bb0bf50 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:33.940518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e5007-c2f6-42ee-89c1-049d399ed791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:33.941182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e5007-c2f6-42ee-89c1-049d399ed791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:33.941456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e5007-c2f6-42ee-89c1-049d399ed791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:33.941724 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e5007-c2f6-42ee-89c1-049d399ed791 admin admin] PATH_INFO: `/v3/users/67d0582228b043c28fbcdb2b60345f21` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:33.942898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e5007-c2f6-42ee-89c1-049d399ed791 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67d0582228b043c28fbcdb2b60345f21)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:33.944820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e5007-c2f6-42ee-89c1-049d399ed791 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:33.993059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc5e5007-c2f6-42ee-89c1-049d399ed791 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67d0582228b043c28fbcdb2b60345f21'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:33.994329 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc5e5007-c2f6-42ee-89c1-049d399ed791 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67d0582228b043c28fbcdb2b60345f21'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:34.001373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc5e5007-c2f6-42ee-89c1-049d399ed791 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67d0582228b043c28fbcdb2b60345f21'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:34.009528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc5e5007-c2f6-42ee-89c1-049d399ed791 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67d0582228b043c28fbcdb2b60345f21'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:34.022331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc5e5007-c2f6-42ee-89c1-049d399ed791 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67d0582228b043c28fbcdb2b60345f21'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:34.028437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1131/2249] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:09:33 2022] DELETE /identity/v3/users/67d0582228b043c28fbcdb2b60345f21 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:34.033301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44598277-3bde-40e0-ad49-b0251915c236 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:34.033768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44598277-3bde-40e0-ad49-b0251915c236 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:34.033886 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44598277-3bde-40e0-ad49-b0251915c236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:34.052633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-44598277-3bde-40e0-ad49-b0251915c236 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:34.053833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-44598277-3bde-40e0-ad49-b0251915c236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:34.111904 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1119/2250] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:34.122420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e5007-c2f6-42ee-89c1-049d399ed791 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:34.123858 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cc5e5007-c2f6-42ee-89c1-049d399ed791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:34.181215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e5007-c2f6-42ee-89c1-049d399ed791 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:34.184827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2e6bb-9dcb-480c-b0b3-9b86872c63a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:34.185419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2e6bb-9dcb-480c-b0b3-9b86872c63a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:34.185687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2e6bb-9dcb-480c-b0b3-9b86872c63a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:34.185952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2e6bb-9dcb-480c-b0b3-9b86872c63a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:34.195432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2e6bb-9dcb-480c-b0b3-9b86872c63a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:34.197449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2e6bb-9dcb-480c-b0b3-9b86872c63a8 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:34.207143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1132/2251] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:09:34.487256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44598277-3bde-40e0-ad49-b0251915c236 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:34.489000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-44598277-3bde-40e0-ad49-b0251915c236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:34.506849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44598277-3bde-40e0-ad49-b0251915c236 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:34.513857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517e6e8-2fe2-42c4-b1bf-b4dc47f82041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:34.514545 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517e6e8-2fe2-42c4-b1bf-b4dc47f82041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:34.514847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517e6e8-2fe2-42c4-b1bf-b4dc47f82041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:34.515113 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517e6e8-2fe2-42c4-b1bf-b4dc47f82041 admin admin] PATH_INFO: `/v3/projects/df92c18e38a3473eb221a063734ec863` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:34.516351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517e6e8-2fe2-42c4-b1bf-b4dc47f82041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df92c18e38a3473eb221a063734ec863)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:34.518288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517e6e8-2fe2-42c4-b1bf-b4dc47f82041 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:34.570363 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2517e6e8-2fe2-42c4-b1bf-b4dc47f82041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df92c18e38a3473eb221a063734ec863', 'request_id': 'req-2517e6e8-2fe2-42c4-b1bf-b4dc47f82041'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:34.579384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2517e6e8-2fe2-42c4-b1bf-b4dc47f82041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df92c18e38a3473eb221a063734ec863', 'request_id': 'req-2517e6e8-2fe2-42c4-b1bf-b4dc47f82041'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:34.586064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2517e6e8-2fe2-42c4-b1bf-b4dc47f82041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df92c18e38a3473eb221a063734ec863', 'request_id': 'req-2517e6e8-2fe2-42c4-b1bf-b4dc47f82041'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:34.599080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2517e6e8-2fe2-42c4-b1bf-b4dc47f82041 admin admin] The token cache is being invalidate because project df92c18e38a3473eb221a063734ec863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:09:34.599369 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2517e6e8-2fe2-42c4-b1bf-b4dc47f82041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:34.600195 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2517e6e8-2fe2-42c4-b1bf-b4dc47f82041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:34.602234 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1120/2252] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:09:34 2022] DELETE /identity/v3/projects/df92c18e38a3473eb221a063734ec863 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:34.608043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e76bf8-22f5-4e16-af58-4a9a4b9136ad None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:34.608398 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e76bf8-22f5-4e16-af58-4a9a4b9136ad None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:34.608669 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e76bf8-22f5-4e16-af58-4a9a4b9136ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:34.631218 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-16e76bf8-22f5-4e16-af58-4a9a4b9136ad None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:34.632454 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-16e76bf8-22f5-4e16-af58-4a9a4b9136ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:34.671719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:34.702685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1133/2253] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:34.710769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884783e0-e976-4d04-97ae-6889f5c0c52f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:34.711061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884783e0-e976-4d04-97ae-6889f5c0c52f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:34.711333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884783e0-e976-4d04-97ae-6889f5c0c52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:34.731947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-884783e0-e976-4d04-97ae-6889f5c0c52f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:34.733186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-884783e0-e976-4d04-97ae-6889f5c0c52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:34.741109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1121/2254] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:34.746725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e76bf8-22f5-4e16-af58-4a9a4b9136ad None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:34.758043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e76bf8-22f5-4e16-af58-4a9a4b9136ad None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:34.766074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba821262-94ec-4218-8205-da3759921ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:34.766074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba821262-94ec-4218-8205-da3759921ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:34.766074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba821262-94ec-4218-8205-da3759921ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:34.766074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba821262-94ec-4218-8205-da3759921ee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:34.766446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba821262-94ec-4218-8205-da3759921ee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:09:34.766446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba821262-94ec-4218-8205-da3759921ee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:34.767084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba821262-94ec-4218-8205-da3759921ee3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:34.786024 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1134/2255] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:09:34.787134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884783e0-e976-4d04-97ae-6889f5c0c52f None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:34.797597 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884783e0-e976-4d04-97ae-6889f5c0c52f None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:34.802653 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11aa2d-0ce5-4228-a8b4-5317df376316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:34.803283 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11aa2d-0ce5-4228-a8b4-5317df376316 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:34.803559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11aa2d-0ce5-4228-a8b4-5317df376316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:34.803829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11aa2d-0ce5-4228-a8b4-5317df376316 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:34.804503 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be11aa2d-0ce5-4228-a8b4-5317df376316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:34.806851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be11aa2d-0ce5-4228-a8b4-5317df376316 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:34.838264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1122/2256] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:09:34 2022] POST /identity/v3/projects => generated 397 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:09:34.839960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba821262-94ec-4218-8205-da3759921ee3 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:34.850305 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba821262-94ec-4218-8205-da3759921ee3 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:34.909853 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfee24-6d02-4a6e-b250-0bea822916bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:34.910568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfee24-6d02-4a6e-b250-0bea822916bd admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:34.910669 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfee24-6d02-4a6e-b250-0bea822916bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:34.910935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfee24-6d02-4a6e-b250-0bea822916bd admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:34.911567 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfee24-6d02-4a6e-b250-0bea822916bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:34.913482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfee24-6d02-4a6e-b250-0bea822916bd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:34.937426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1135/2257] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:09:34 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:09:34.943470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11aa2d-0ce5-4228-a8b4-5317df376316 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:34.957040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11aa2d-0ce5-4228-a8b4-5317df376316 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:34.961841 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9aa3f-515c-42b3-8132-fed5e579ed30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:34.962976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9aa3f-515c-42b3-8132-fed5e579ed30 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:34.963255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9aa3f-515c-42b3-8132-fed5e579ed30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:34.963529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9aa3f-515c-42b3-8132-fed5e579ed30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:34.964176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9aa3f-515c-42b3-8132-fed5e579ed30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:34.965784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9aa3f-515c-42b3-8132-fed5e579ed30 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:34.975856 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1123/2258] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:34.980669 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfee24-6d02-4a6e-b250-0bea822916bd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:34.990895 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfee24-6d02-4a6e-b250-0bea822916bd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:34.995881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d53bd6-3fae-4564-8e39-09c84fdb53a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:34.996474 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d53bd6-3fae-4564-8e39-09c84fdb53a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:34.996754 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d53bd6-3fae-4564-8e39-09c84fdb53a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:34.997027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d53bd6-3fae-4564-8e39-09c84fdb53a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:34.997713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d53bd6-3fae-4564-8e39-09c84fdb53a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:34.999336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d53bd6-3fae-4564-8e39-09c84fdb53a9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:35.009207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1136/2259] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:35.013506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9aa3f-515c-42b3-8132-fed5e579ed30 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:35.023789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9aa3f-515c-42b3-8132-fed5e579ed30 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:35.030709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153db65-ff70-4559-bcf2-b8a320fb5c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:35.031375 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153db65-ff70-4559-bcf2-b8a320fb5c24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:35.031656 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153db65-ff70-4559-bcf2-b8a320fb5c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:35.031931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153db65-ff70-4559-bcf2-b8a320fb5c24 admin admin] PATH_INFO: `/v3/projects/890fe2cbaaaf4defb5290343363d47fa/users/3038265fb6744748b5959d84f5db6dad/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:35.049058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153db65-ff70-4559-bcf2-b8a320fb5c24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=890fe2cbaaaf4defb5290343363d47fa, user_id=3038265fb6744748b5959d84f5db6dad, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:35.051831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153db65-ff70-4559-bcf2-b8a320fb5c24 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:35.065346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1124/2260] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:09:35 2022] PUT /identity/v3/projects/890fe2cbaaaf4defb5290343363d47fa/users/3038265fb6744748b5959d84f5db6dad/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:35.075748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdae73-dcd1-4d27-925a-60515155fde9 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:35.076043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdae73-dcd1-4d27-925a-60515155fde9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:35.076404 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdae73-dcd1-4d27-925a-60515155fde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:35.106759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a8fdae73-dcd1-4d27-925a-60515155fde9 None None] MFA Rules not processed for user `3038265fb6744748b5959d84f5db6dad`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:35.107969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a8fdae73-dcd1-4d27-925a-60515155fde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:35.181625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1137/2261] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:35 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:35.192962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153db65-ff70-4559-bcf2-b8a320fb5c24 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:35.194710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-5153db65-ff70-4559-bcf2-b8a320fb5c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:35.216258 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:35.257604 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153db65-ff70-4559-bcf2-b8a320fb5c24 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:35.261845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1de8f-43e7-46cf-bcaa-1b62eea69a3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:35.262478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1de8f-43e7-46cf-bcaa-1b62eea69a3a service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:35.262751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1de8f-43e7-46cf-bcaa-1b62eea69a3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:35.263018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1de8f-43e7-46cf-bcaa-1b62eea69a3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:35.271306 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1de8f-43e7-46cf-bcaa-1b62eea69a3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:35.272724 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1de8f-43e7-46cf-bcaa-1b62eea69a3a service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:35.282211 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1125/2262] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:35 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:09:35.892911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fdae73-dcd1-4d27-925a-60515155fde9 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:35.894400 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a8fdae73-dcd1-4d27-925a-60515155fde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:35.934780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:35.960444 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fdae73-dcd1-4d27-925a-60515155fde9 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:35.964883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93637b46-0da5-4ef7-8563-55690e45fdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:35.965506 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93637b46-0da5-4ef7-8563-55690e45fdbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:35.965784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93637b46-0da5-4ef7-8563-55690e45fdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:35.966056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93637b46-0da5-4ef7-8563-55690e45fdbf admin admin] PATH_INFO: `/v3/users/8f7f35ed4dab44c6adf390da3649a0bd` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:35.967465 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93637b46-0da5-4ef7-8563-55690e45fdbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f7f35ed4dab44c6adf390da3649a0bd)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:35.969370 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93637b46-0da5-4ef7-8563-55690e45fdbf admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:36.017505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-93637b46-0da5-4ef7-8563-55690e45fdbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f7f35ed4dab44c6adf390da3649a0bd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:36.018198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-93637b46-0da5-4ef7-8563-55690e45fdbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f7f35ed4dab44c6adf390da3649a0bd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:36.025651 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-93637b46-0da5-4ef7-8563-55690e45fdbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f7f35ed4dab44c6adf390da3649a0bd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:36.033795 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-93637b46-0da5-4ef7-8563-55690e45fdbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f7f35ed4dab44c6adf390da3649a0bd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:36.047873 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-93637b46-0da5-4ef7-8563-55690e45fdbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f7f35ed4dab44c6adf390da3649a0bd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:36.053928 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1138/2263] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:09:35 2022] DELETE /identity/v3/users/8f7f35ed4dab44c6adf390da3649a0bd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:36.059062 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684030d-07d2-4b7e-99e8-0c03e78ce170 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:36.059556 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684030d-07d2-4b7e-99e8-0c03e78ce170 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:36.059674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684030d-07d2-4b7e-99e8-0c03e78ce170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:36.078621 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f684030d-07d2-4b7e-99e8-0c03e78ce170 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:36.079839 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f684030d-07d2-4b7e-99e8-0c03e78ce170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:36.138776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1126/2264] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:36.148840 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93637b46-0da5-4ef7-8563-55690e45fdbf admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:36.150377 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-93637b46-0da5-4ef7-8563-55690e45fdbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:36.207658 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93637b46-0da5-4ef7-8563-55690e45fdbf admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:36.211339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417d56d-5c8b-491f-9650-5591ea795301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:36.211889 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417d56d-5c8b-491f-9650-5591ea795301 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:36.212169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417d56d-5c8b-491f-9650-5591ea795301 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:36.212436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417d56d-5c8b-491f-9650-5591ea795301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:36.224530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417d56d-5c8b-491f-9650-5591ea795301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:36.225945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417d56d-5c8b-491f-9650-5591ea795301 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:36.238093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1139/2265] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:09:36.509846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684030d-07d2-4b7e-99e8-0c03e78ce170 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:36.511356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f684030d-07d2-4b7e-99e8-0c03e78ce170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:36.524265 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684030d-07d2-4b7e-99e8-0c03e78ce170 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:36.530016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c535a3-407d-47a8-abcf-2eef6517f4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:36.531209 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c535a3-407d-47a8-abcf-2eef6517f4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:36.531658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c535a3-407d-47a8-abcf-2eef6517f4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:36.531935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c535a3-407d-47a8-abcf-2eef6517f4f9 admin admin] PATH_INFO: `/v3/projects/671ac82e302b48c4a32bb1e65dca4b61` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:36.533209 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c535a3-407d-47a8-abcf-2eef6517f4f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=671ac82e302b48c4a32bb1e65dca4b61)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:36.535516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c535a3-407d-47a8-abcf-2eef6517f4f9 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:36.582426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f4c535a3-407d-47a8-abcf-2eef6517f4f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '671ac82e302b48c4a32bb1e65dca4b61', 'request_id': 'req-f4c535a3-407d-47a8-abcf-2eef6517f4f9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:36.590351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f4c535a3-407d-47a8-abcf-2eef6517f4f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '671ac82e302b48c4a32bb1e65dca4b61', 'request_id': 'req-f4c535a3-407d-47a8-abcf-2eef6517f4f9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:36.596514 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f4c535a3-407d-47a8-abcf-2eef6517f4f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '671ac82e302b48c4a32bb1e65dca4b61', 'request_id': 'req-f4c535a3-407d-47a8-abcf-2eef6517f4f9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:36.607911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f4c535a3-407d-47a8-abcf-2eef6517f4f9 admin admin] The token cache is being invalidate because project 671ac82e302b48c4a32bb1e65dca4b61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:09:36.608195 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f4c535a3-407d-47a8-abcf-2eef6517f4f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:36.608831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f4c535a3-407d-47a8-abcf-2eef6517f4f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:36.610359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1127/2266] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:09:36 2022] DELETE /identity/v3/projects/671ac82e302b48c4a32bb1e65dca4b61 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:36.614480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417d56d-5c8b-491f-9650-5591ea795301 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:36.624080 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417d56d-5c8b-491f-9650-5591ea795301 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:36.674399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:36.681328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b584b7b-9acb-478c-bb1b-69b8b47a43e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:36.682154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b584b7b-9acb-478c-bb1b-69b8b47a43e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:36.682580 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b584b7b-9acb-478c-bb1b-69b8b47a43e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:36.683018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b584b7b-9acb-478c-bb1b-69b8b47a43e8 admin admin] PATH_INFO: `/v3/users/97e42239729343f58bccef35a490842c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:36.684313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b584b7b-9acb-478c-bb1b-69b8b47a43e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97e42239729343f58bccef35a490842c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:36.686225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b584b7b-9acb-478c-bb1b-69b8b47a43e8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:36.739354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3b584b7b-9acb-478c-bb1b-69b8b47a43e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97e42239729343f58bccef35a490842c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:36.740088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3b584b7b-9acb-478c-bb1b-69b8b47a43e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97e42239729343f58bccef35a490842c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:36.747966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3b584b7b-9acb-478c-bb1b-69b8b47a43e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97e42239729343f58bccef35a490842c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:36.755923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3b584b7b-9acb-478c-bb1b-69b8b47a43e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97e42239729343f58bccef35a490842c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:36.778026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3b584b7b-9acb-478c-bb1b-69b8b47a43e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97e42239729343f58bccef35a490842c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:36.784984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1140/2267] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:09:36 2022] DELETE /identity/v3/users/97e42239729343f58bccef35a490842c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:37.086353 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c535a3-407d-47a8-abcf-2eef6517f4f9 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:37.088177 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f4c535a3-407d-47a8-abcf-2eef6517f4f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:37.103298 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c535a3-407d-47a8-abcf-2eef6517f4f9 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:37.148197 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:37.164760 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702cffa-cff6-41b2-bd21-3c6e86594028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:37.165452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702cffa-cff6-41b2-bd21-3c6e86594028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:37.165730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702cffa-cff6-41b2-bd21-3c6e86594028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:37.166000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702cffa-cff6-41b2-bd21-3c6e86594028 admin admin] PATH_INFO: `/v3/projects/bbdbc7cb60ef4cfd94e127e650a4dcc0` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:37.167198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702cffa-cff6-41b2-bd21-3c6e86594028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbdbc7cb60ef4cfd94e127e650a4dcc0)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:37.169147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702cffa-cff6-41b2-bd21-3c6e86594028 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:37.215671 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0702cffa-cff6-41b2-bd21-3c6e86594028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbdbc7cb60ef4cfd94e127e650a4dcc0', 'request_id': 'req-0702cffa-cff6-41b2-bd21-3c6e86594028'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:37.226065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0702cffa-cff6-41b2-bd21-3c6e86594028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbdbc7cb60ef4cfd94e127e650a4dcc0', 'request_id': 'req-0702cffa-cff6-41b2-bd21-3c6e86594028'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:37.232637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0702cffa-cff6-41b2-bd21-3c6e86594028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbdbc7cb60ef4cfd94e127e650a4dcc0', 'request_id': 'req-0702cffa-cff6-41b2-bd21-3c6e86594028'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:37.244786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0702cffa-cff6-41b2-bd21-3c6e86594028 admin admin] The token cache is being invalidate because project bbdbc7cb60ef4cfd94e127e650a4dcc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:09:37.245077 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0702cffa-cff6-41b2-bd21-3c6e86594028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:37.246037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0702cffa-cff6-41b2-bd21-3c6e86594028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:37.247584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1128/2268] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:09:37 2022] DELETE /identity/v3/projects/bbdbc7cb60ef4cfd94e127e650a4dcc0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:09:37.252410 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b584b7b-9acb-478c-bb1b-69b8b47a43e8 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:37.254225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3b584b7b-9acb-478c-bb1b-69b8b47a43e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:37.269839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b584b7b-9acb-478c-bb1b-69b8b47a43e8 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:37.328771 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d2ac6-73f4-4ad7-8146-bd3fdf043e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:37.329349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d2ac6-73f4-4ad7-8146-bd3fdf043e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:37.329629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d2ac6-73f4-4ad7-8146-bd3fdf043e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:37.329903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d2ac6-73f4-4ad7-8146-bd3fdf043e92 admin admin] PATH_INFO: `/v3/users/b2ea893959d74e88b10ba220a4ca6ae5` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:37.331088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d2ac6-73f4-4ad7-8146-bd3fdf043e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ea893959d74e88b10ba220a4ca6ae5)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:37.333015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d2ac6-73f4-4ad7-8146-bd3fdf043e92 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:37.380319 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-795d2ac6-73f4-4ad7-8146-bd3fdf043e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ea893959d74e88b10ba220a4ca6ae5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:37.380960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-795d2ac6-73f4-4ad7-8146-bd3fdf043e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ea893959d74e88b10ba220a4ca6ae5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:37.388696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-795d2ac6-73f4-4ad7-8146-bd3fdf043e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ea893959d74e88b10ba220a4ca6ae5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:37.396161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-795d2ac6-73f4-4ad7-8146-bd3fdf043e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ea893959d74e88b10ba220a4ca6ae5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:37.407277 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:37.408460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-795d2ac6-73f4-4ad7-8146-bd3fdf043e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ea893959d74e88b10ba220a4ca6ae5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:37.417006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1141/2269] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:09:37 2022] DELETE /identity/v3/users/b2ea893959d74e88b10ba220a4ca6ae5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:09:37.692243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702cffa-cff6-41b2-bd21-3c6e86594028 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:37.693963 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0702cffa-cff6-41b2-bd21-3c6e86594028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:37.709234 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702cffa-cff6-41b2-bd21-3c6e86594028 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:37.767962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab65774-0c93-4d3f-a396-471b28f0ed02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:37.768593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab65774-0c93-4d3f-a396-471b28f0ed02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:37.768872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab65774-0c93-4d3f-a396-471b28f0ed02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:37.769144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab65774-0c93-4d3f-a396-471b28f0ed02 admin admin] PATH_INFO: `/v3/projects/70f2600f735440b1adf201ae1f05ddc7` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:37.770321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab65774-0c93-4d3f-a396-471b28f0ed02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70f2600f735440b1adf201ae1f05ddc7)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:37.772232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab65774-0c93-4d3f-a396-471b28f0ed02 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:37.818221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7ab65774-0c93-4d3f-a396-471b28f0ed02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70f2600f735440b1adf201ae1f05ddc7', 'request_id': 'req-7ab65774-0c93-4d3f-a396-471b28f0ed02'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:37.827510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7ab65774-0c93-4d3f-a396-471b28f0ed02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70f2600f735440b1adf201ae1f05ddc7', 'request_id': 'req-7ab65774-0c93-4d3f-a396-471b28f0ed02'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:37.833203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7ab65774-0c93-4d3f-a396-471b28f0ed02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70f2600f735440b1adf201ae1f05ddc7', 'request_id': 'req-7ab65774-0c93-4d3f-a396-471b28f0ed02'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:37.844453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7ab65774-0c93-4d3f-a396-471b28f0ed02 admin admin] The token cache is being invalidate because project 70f2600f735440b1adf201ae1f05ddc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:09:37.844738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7ab65774-0c93-4d3f-a396-471b28f0ed02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:37.845390 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7ab65774-0c93-4d3f-a396-471b28f0ed02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:37.846935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1129/2270] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:09:37 2022] DELETE /identity/v3/projects/70f2600f735440b1adf201ae1f05ddc7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:37.851710 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75102e-2d4d-492e-8785-3040b41cb72e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:37.851999 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75102e-2d4d-492e-8785-3040b41cb72e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:37.852276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75102e-2d4d-492e-8785-3040b41cb72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:37.872577 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-fa75102e-2d4d-492e-8785-3040b41cb72e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:37.873708 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fa75102e-2d4d-492e-8785-3040b41cb72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:37.933002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1142/2271] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:37.940977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadb85c-d30a-42b3-807b-7b1bc9f569ee None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:37.941263 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadb85c-d30a-42b3-807b-7b1bc9f569ee None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:37.941542 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadb85c-d30a-42b3-807b-7b1bc9f569ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:37.950787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:37.962752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-ecadb85c-d30a-42b3-807b-7b1bc9f569ee None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:37.964186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ecadb85c-d30a-42b3-807b-7b1bc9f569ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:37.972163 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1130/2272] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:37.976403 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75102e-2d4d-492e-8785-3040b41cb72e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:37.986425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75102e-2d4d-492e-8785-3040b41cb72e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:37.991173 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f364b-3ca4-4f62-bac3-905fa527c832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:37.991733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f364b-3ca4-4f62-bac3-905fa527c832 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:37.992083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f364b-3ca4-4f62-bac3-905fa527c832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:37.992311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f364b-3ca4-4f62-bac3-905fa527c832 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:37.992910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f364b-3ca4-4f62-bac3-905fa527c832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:09:37.993788 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f364b-3ca4-4f62-bac3-905fa527c832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:38.000781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f364b-3ca4-4f62-bac3-905fa527c832 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:38.008574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1143/2273] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:09:38.013577 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadb85c-d30a-42b3-807b-7b1bc9f569ee None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:38.023422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadb85c-d30a-42b3-807b-7b1bc9f569ee None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:38.028205 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bed7f9-3b25-42c2-9997-09058ec7a79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:38.028804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bed7f9-3b25-42c2-9997-09058ec7a79c admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:38.029084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bed7f9-3b25-42c2-9997-09058ec7a79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:38.029358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bed7f9-3b25-42c2-9997-09058ec7a79c admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:38.029997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bed7f9-3b25-42c2-9997-09058ec7a79c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:38.031941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bed7f9-3b25-42c2-9997-09058ec7a79c admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:38.060353 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1131/2274] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:09:38 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:09:38.064575 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f364b-3ca4-4f62-bac3-905fa527c832 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:38.075617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f364b-3ca4-4f62-bac3-905fa527c832 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:38.133536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ffd69-b53a-4566-9c30-9f51db5beb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:38.134740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ffd69-b53a-4566-9c30-9f51db5beb27 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:38.135016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ffd69-b53a-4566-9c30-9f51db5beb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:38.135280 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ffd69-b53a-4566-9c30-9f51db5beb27 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:38.135913 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ffd69-b53a-4566-9c30-9f51db5beb27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:38.137830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ffd69-b53a-4566-9c30-9f51db5beb27 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:38.158237 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1144/2275] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:09:38 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:09:38.162940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bed7f9-3b25-42c2-9997-09058ec7a79c admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:38.174604 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bed7f9-3b25-42c2-9997-09058ec7a79c admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:38.180260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1180b-5ed9-4d92-8aaf-290d00fd836e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:38.181546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1180b-5ed9-4d92-8aaf-290d00fd836e admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:38.181984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1180b-5ed9-4d92-8aaf-290d00fd836e admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:38.182204 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1180b-5ed9-4d92-8aaf-290d00fd836e admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:38.182805 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1180b-5ed9-4d92-8aaf-290d00fd836e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:38.184407 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1180b-5ed9-4d92-8aaf-290d00fd836e admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:38.194794 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1132/2276] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:09:38.198970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ffd69-b53a-4566-9c30-9f51db5beb27 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:38.209312 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ffd69-b53a-4566-9c30-9f51db5beb27 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:38.214345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b6f97-4b9b-4457-bab0-6684dce9bda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:38.214950 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b6f97-4b9b-4457-bab0-6684dce9bda4 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:38.215225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b6f97-4b9b-4457-bab0-6684dce9bda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:38.215497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b6f97-4b9b-4457-bab0-6684dce9bda4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:38.216138 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b6f97-4b9b-4457-bab0-6684dce9bda4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:38.217733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b6f97-4b9b-4457-bab0-6684dce9bda4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:38.230504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1145/2277] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:09:38.234950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1180b-5ed9-4d92-8aaf-290d00fd836e admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:38.246039 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1180b-5ed9-4d92-8aaf-290d00fd836e admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:38.250768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d4ce1-701b-48f5-88f4-32daee69d345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:38.251409 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d4ce1-701b-48f5-88f4-32daee69d345 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:38.251689 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d4ce1-701b-48f5-88f4-32daee69d345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:38.251960 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d4ce1-701b-48f5-88f4-32daee69d345 admin admin] PATH_INFO: `/v3/projects/384f8901efef448bb9ad0dbfeee24917/users/908dff5336054538865950ae38263e54/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:38.266376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d4ce1-701b-48f5-88f4-32daee69d345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=384f8901efef448bb9ad0dbfeee24917, user_id=908dff5336054538865950ae38263e54, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:38.269012 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d4ce1-701b-48f5-88f4-32daee69d345 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:38.281350 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1133/2278] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:09:38 2022] PUT /identity/v3/projects/384f8901efef448bb9ad0dbfeee24917/users/908dff5336054538865950ae38263e54/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:38.349185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ee05c-1833-47b7-94d8-ed49d9a0cf51 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:38.349767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ee05c-1833-47b7-94d8-ed49d9a0cf51 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:38.349767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ee05c-1833-47b7-94d8-ed49d9a0cf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:38.380269 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4b8ee05c-1833-47b7-94d8-ed49d9a0cf51 None None] MFA Rules not processed for user `908dff5336054538865950ae38263e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:38.381604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4b8ee05c-1833-47b7-94d8-ed49d9a0cf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:38.428154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:38.446261 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1146/2279] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:38.456939 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d4ce1-701b-48f5-88f4-32daee69d345 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:38.458641 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e72d4ce1-701b-48f5-88f4-32daee69d345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:38.514669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d4ce1-701b-48f5-88f4-32daee69d345 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:38.518295 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540edc8-8567-4f88-80bc-7b7845dfc6ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:38.518890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540edc8-8567-4f88-80bc-7b7845dfc6ce service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:38.519168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540edc8-8567-4f88-80bc-7b7845dfc6ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:38.519433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540edc8-8567-4f88-80bc-7b7845dfc6ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:38.528147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540edc8-8567-4f88-80bc-7b7845dfc6ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:38.529564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540edc8-8567-4f88-80bc-7b7845dfc6ce service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:38.538197 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1134/2280] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:38 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:09:48.443339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:48.534280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:48.635485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ee05c-1833-47b7-94d8-ed49d9a0cf51 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:48.636900 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4b8ee05c-1833-47b7-94d8-ed49d9a0cf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:48.701684 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ee05c-1833-47b7-94d8-ed49d9a0cf51 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:48.706242 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c73b29-e186-45dd-9124-7d6753454de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:48.706868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c73b29-e186-45dd-9124-7d6753454de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:48.707146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c73b29-e186-45dd-9124-7d6753454de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:48.707418 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c73b29-e186-45dd-9124-7d6753454de8 admin admin] PATH_INFO: `/v3/users/3038265fb6744748b5959d84f5db6dad` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:48.708534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c73b29-e186-45dd-9124-7d6753454de8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3038265fb6744748b5959d84f5db6dad)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:48.710480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c73b29-e186-45dd-9124-7d6753454de8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:48.756420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e4c73b29-e186-45dd-9124-7d6753454de8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3038265fb6744748b5959d84f5db6dad'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:48.757008 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e4c73b29-e186-45dd-9124-7d6753454de8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3038265fb6744748b5959d84f5db6dad'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:48.764266 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e4c73b29-e186-45dd-9124-7d6753454de8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3038265fb6744748b5959d84f5db6dad'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:48.771735 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e4c73b29-e186-45dd-9124-7d6753454de8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3038265fb6744748b5959d84f5db6dad'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:48.783743 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e4c73b29-e186-45dd-9124-7d6753454de8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3038265fb6744748b5959d84f5db6dad'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:48.789766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1147/2281] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:09:48 2022] DELETE /identity/v3/users/3038265fb6744748b5959d84f5db6dad => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:48.794804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b927a4-4758-49de-9cb5-7aa186c21023 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:48.795091 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b927a4-4758-49de-9cb5-7aa186c21023 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:48.795367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b927a4-4758-49de-9cb5-7aa186c21023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:48.815420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-98b927a4-4758-49de-9cb5-7aa186c21023 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:48.816635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-98b927a4-4758-49de-9cb5-7aa186c21023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:48.875688 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1135/2282] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:48.885490 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c73b29-e186-45dd-9124-7d6753454de8 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:48.887055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e4c73b29-e186-45dd-9124-7d6753454de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:48.942764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c73b29-e186-45dd-9124-7d6753454de8 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:48.946833 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb98ab-7cce-439a-ad20-46e5ddc3dcde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:48.947428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb98ab-7cce-439a-ad20-46e5ddc3dcde service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:48.947701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb98ab-7cce-439a-ad20-46e5ddc3dcde service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:48.947970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb98ab-7cce-439a-ad20-46e5ddc3dcde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:48.956857 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb98ab-7cce-439a-ad20-46e5ddc3dcde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:48.958282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb98ab-7cce-439a-ad20-46e5ddc3dcde service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:48.968131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1148/2283] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:09:49.235703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b927a4-4758-49de-9cb5-7aa186c21023 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:49.237116 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-98b927a4-4758-49de-9cb5-7aa186c21023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:49.252520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b927a4-4758-49de-9cb5-7aa186c21023 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:49.257310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a0ce9-d9a3-402a-8e88-b08aeec33fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:49.257963 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a0ce9-d9a3-402a-8e88-b08aeec33fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:49.258281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a0ce9-d9a3-402a-8e88-b08aeec33fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:49.258553 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a0ce9-d9a3-402a-8e88-b08aeec33fb2 admin admin] PATH_INFO: `/v3/projects/890fe2cbaaaf4defb5290343363d47fa` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:49.260134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a0ce9-d9a3-402a-8e88-b08aeec33fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=890fe2cbaaaf4defb5290343363d47fa)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:49.262858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a0ce9-d9a3-402a-8e88-b08aeec33fb2 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:49.317452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-616a0ce9-d9a3-402a-8e88-b08aeec33fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '890fe2cbaaaf4defb5290343363d47fa', 'request_id': 'req-616a0ce9-d9a3-402a-8e88-b08aeec33fb2'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:49.325967 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-616a0ce9-d9a3-402a-8e88-b08aeec33fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '890fe2cbaaaf4defb5290343363d47fa', 'request_id': 'req-616a0ce9-d9a3-402a-8e88-b08aeec33fb2'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:49.331837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-616a0ce9-d9a3-402a-8e88-b08aeec33fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '890fe2cbaaaf4defb5290343363d47fa', 'request_id': 'req-616a0ce9-d9a3-402a-8e88-b08aeec33fb2'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:49.343498 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-616a0ce9-d9a3-402a-8e88-b08aeec33fb2 admin admin] The token cache is being invalidate because project 890fe2cbaaaf4defb5290343363d47fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:09:49.343787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-616a0ce9-d9a3-402a-8e88-b08aeec33fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:49.344462 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-616a0ce9-d9a3-402a-8e88-b08aeec33fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:49.345960 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1136/2284] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:09:49 2022] DELETE /identity/v3/projects/890fe2cbaaaf4defb5290343363d47fa => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:58.963390 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:59.339854 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:10:05.119804 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb98ab-7cce-439a-ad20-46e5ddc3dcde service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:10:05.121885 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d8bb98ab-7cce-439a-ad20-46e5ddc3dcde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:10:05.135459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb98ab-7cce-439a-ad20-46e5ddc3dcde service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:10:05.195860 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f2580-d845-495d-8aea-4180c296f7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:10:05.196502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f2580-d845-495d-8aea-4180c296f7bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:05.196779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f2580-d845-495d-8aea-4180c296f7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:05.197047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f2580-d845-495d-8aea-4180c296f7bf admin admin] PATH_INFO: `/v3/users/908dff5336054538865950ae38263e54` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:05.198230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f2580-d845-495d-8aea-4180c296f7bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=908dff5336054538865950ae38263e54)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:10:05.200609 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f2580-d845-495d-8aea-4180c296f7bf admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:10:05.247480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d79f2580-d845-495d-8aea-4180c296f7bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '908dff5336054538865950ae38263e54'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:10:05.248128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d79f2580-d845-495d-8aea-4180c296f7bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '908dff5336054538865950ae38263e54'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:10:05.254956 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d79f2580-d845-495d-8aea-4180c296f7bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '908dff5336054538865950ae38263e54'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:10:05.262785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d79f2580-d845-495d-8aea-4180c296f7bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '908dff5336054538865950ae38263e54'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:10:05.274839 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d79f2580-d845-495d-8aea-4180c296f7bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '908dff5336054538865950ae38263e54'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:10:05.281643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1149/2285] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:10:05 2022] DELETE /identity/v3/users/908dff5336054538865950ae38263e54 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:10:05.286080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d5d24-bfe7-4ed0-9b2f-405f3e1c1051 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:05.286576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d5d24-bfe7-4ed0-9b2f-405f3e1c1051 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:05.286724 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d5d24-bfe7-4ed0-9b2f-405f3e1c1051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:05.305403 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-261d5d24-bfe7-4ed0-9b2f-405f3e1c1051 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:10:05.306636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-261d5d24-bfe7-4ed0-9b2f-405f3e1c1051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:10:05.364804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1137/2286] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:10:05.375897 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f2580-d845-495d-8aea-4180c296f7bf admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:10:05.378218 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d79f2580-d845-495d-8aea-4180c296f7bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:10:05.439203 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f2580-d845-495d-8aea-4180c296f7bf admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:10:05.443392 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7db6b5-75d2-4a18-8ee4-cb78e9e11368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:10:05.444011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7db6b5-75d2-4a18-8ee4-cb78e9e11368 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:05.444282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7db6b5-75d2-4a18-8ee4-cb78e9e11368 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:05.444571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7db6b5-75d2-4a18-8ee4-cb78e9e11368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:05.457440 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7db6b5-75d2-4a18-8ee4-cb78e9e11368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:10:05.459421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7db6b5-75d2-4a18-8ee4-cb78e9e11368 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:10:05.469712 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1150/2287] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:10:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:10:05.738084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d5d24-bfe7-4ed0-9b2f-405f3e1c1051 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:10:05.739559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-261d5d24-bfe7-4ed0-9b2f-405f3e1c1051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:10:05.753543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d5d24-bfe7-4ed0-9b2f-405f3e1c1051 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:10:05.758521 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2efb7-5597-416b-a22b-293c179d80c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:10:05.759169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2efb7-5597-416b-a22b-293c179d80c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:05.759449 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2efb7-5597-416b-a22b-293c179d80c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:05.759727 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2efb7-5597-416b-a22b-293c179d80c4 admin admin] PATH_INFO: `/v3/projects/384f8901efef448bb9ad0dbfeee24917` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:05.760938 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2efb7-5597-416b-a22b-293c179d80c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=384f8901efef448bb9ad0dbfeee24917)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:10:05.764543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2efb7-5597-416b-a22b-293c179d80c4 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:10:05.820554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5fb2efb7-5597-416b-a22b-293c179d80c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '384f8901efef448bb9ad0dbfeee24917', 'request_id': 'req-5fb2efb7-5597-416b-a22b-293c179d80c4'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:10:05.828010 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5fb2efb7-5597-416b-a22b-293c179d80c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '384f8901efef448bb9ad0dbfeee24917', 'request_id': 'req-5fb2efb7-5597-416b-a22b-293c179d80c4'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:10:05.833801 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5fb2efb7-5597-416b-a22b-293c179d80c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '384f8901efef448bb9ad0dbfeee24917', 'request_id': 'req-5fb2efb7-5597-416b-a22b-293c179d80c4'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:10:05.844831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5fb2efb7-5597-416b-a22b-293c179d80c4 admin admin] The token cache is being invalidate because project 384f8901efef448bb9ad0dbfeee24917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:10:05.845119 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5fb2efb7-5597-416b-a22b-293c179d80c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:10:05.845751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5fb2efb7-5597-416b-a22b-293c179d80c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:10:05.847265 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1138/2288] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:10:05 2022] DELETE /identity/v3/projects/384f8901efef448bb9ad0dbfeee24917 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:10:15.465139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:10:15.841881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:10:30.500240 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5bfd9-c747-4a4d-8ff1-1dcb581f04bd None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:30.500805 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5bfd9-c747-4a4d-8ff1-1dcb581f04bd None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:30.500805 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5bfd9-c747-4a4d-8ff1-1dcb581f04bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:30.519906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-72f5bfd9-c747-4a4d-8ff1-1dcb581f04bd None None] MFA Rules not processed for user `16c99b5db9794dff83ea1b67fd238655`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:10:30.521116 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-72f5bfd9-c747-4a4d-8ff1-1dcb581f04bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:10:30.575051 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1151/2289] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:10:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:10:40.568453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=25 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:10:49.542594 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b3efc-87e7-4021-b0d3-f6e138c6ddb5 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:49.543099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b3efc-87e7-4021-b0d3-f6e138c6ddb5 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:49.543194 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b3efc-87e7-4021-b0d3-f6e138c6ddb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:49.563327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-bd4b3efc-87e7-4021-b0d3-f6e138c6ddb5 None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:10:49.564539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-bd4b3efc-87e7-4021-b0d3-f6e138c6ddb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:10:49.627908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1139/2290] 10.222.0.62 () {58 vars in 1009 bytes} [Fri Sep 9 12:10:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:10:49.684528 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5bfd9-c747-4a4d-8ff1-1dcb581f04bd None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:10:49.685985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-72f5bfd9-c747-4a4d-8ff1-1dcb581f04bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:10:49.760982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5bfd9-c747-4a4d-8ff1-1dcb581f04bd None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:10:49.766662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bd7ff-7043-416a-b0b8-66b0206b4665 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:10:49.767268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bd7ff-7043-416a-b0b8-66b0206b4665 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:49.767573 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bd7ff-7043-416a-b0b8-66b0206b4665 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:49.767865 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bd7ff-7043-416a-b0b8-66b0206b4665 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:49.776588 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bd7ff-7043-416a-b0b8-66b0206b4665 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:10:49.778845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bd7ff-7043-416a-b0b8-66b0206b4665 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:10:49.787598 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1152/2291] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:10:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:10:59.620477 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=25 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:10:59.784252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=32 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:11:03.944372 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb084ca-4154-4a90-a4e0-22a7c77210c0 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:03.944901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb084ca-4154-4a90-a4e0-22a7c77210c0 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:03.944901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb084ca-4154-4a90-a4e0-22a7c77210c0 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:03.946507 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1140/2292] 10.222.0.62 () {54 vars in 921 bytes} [Fri Sep 9 12:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:11:03.953114 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af134622-5574-42d8-abcd-737f83bcdaf3 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:03.953652 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af134622-5574-42d8-abcd-737f83bcdaf3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:03.953893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af134622-5574-42d8-abcd-737f83bcdaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:03.975670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-af134622-5574-42d8-abcd-737f83bcdaf3 None None] MFA Rules not processed for user `1b30e80c421245b5a1ed5849b28ea90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:11:03.977156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-af134622-5574-42d8-abcd-737f83bcdaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:04.046988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1153/2293] 10.222.0.62 () {58 vars in 1018 bytes} [Fri Sep 9 12:11:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:11:04.059135 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb084ca-4154-4a90-a4e0-22a7c77210c0 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:11:04.060888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0fb084ca-4154-4a90-a4e0-22a7c77210c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:04.152035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb084ca-4154-4a90-a4e0-22a7c77210c0 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:11:04.156427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe11be1-2aaa-4330-b9e4-bc53d7ddf559 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d477a49a1724b7e8a4699a5ebd04bde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:11:04.157027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe11be1-2aaa-4330-b9e4-bc53d7ddf559 service glance] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:04.157305 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe11be1-2aaa-4330-b9e4-bc53d7ddf559 service glance] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:04.157581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe11be1-2aaa-4330-b9e4-bc53d7ddf559 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:04.168830 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe11be1-2aaa-4330-b9e4-bc53d7ddf559 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:11:04.170286 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe11be1-2aaa-4330-b9e4-bc53d7ddf559 service glance] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:11:04.180961 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1141/2294] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:11:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:11:04.255050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162dd95-f42c-432b-a014-8d747773ca5e None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:04.255458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162dd95-f42c-432b-a014-8d747773ca5e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:04.255729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162dd95-f42c-432b-a014-8d747773ca5e None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:04.257083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1154/2295] 10.222.0.62 () {54 vars in 921 bytes} [Fri Sep 9 12:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:11:04.261891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0655522-dc70-43e2-a90f-e8e9318a31dd None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:04.262208 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0655522-dc70-43e2-a90f-e8e9318a31dd None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:04.262484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0655522-dc70-43e2-a90f-e8e9318a31dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:04.282386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d0655522-dc70-43e2-a90f-e8e9318a31dd None None] MFA Rules not processed for user `d00634cdd3fb41e794d3af0b88ba7338`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:11:04.283574 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d0655522-dc70-43e2-a90f-e8e9318a31dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:04.333985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1142/2296] 10.222.0.62 () {58 vars in 1018 bytes} [Fri Sep 9 12:11:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:11:04.346707 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162dd95-f42c-432b-a014-8d747773ca5e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:11:04.348420 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7162dd95-f42c-432b-a014-8d747773ca5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:04.362178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162dd95-f42c-432b-a014-8d747773ca5e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:11:04.365682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7649b11-4e25-4fac-8c11-a5c54cdd77c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:11:04.366300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7649b11-4e25-4fac-8c11-a5c54cdd77c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:04.366578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7649b11-4e25-4fac-8c11-a5c54cdd77c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:04.366847 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7649b11-4e25-4fac-8c11-a5c54cdd77c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:04.374556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7649b11-4e25-4fac-8c11-a5c54cdd77c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:11:04.375993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7649b11-4e25-4fac-8c11-a5c54cdd77c5 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:11:04.383613 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1155/2297] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:11:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:11:04.465604 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85f0e3-c3ac-49d3-a3b2-2c03a56baa79 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:04.465916 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85f0e3-c3ac-49d3-a3b2-2c03a56baa79 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:04.466218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85f0e3-c3ac-49d3-a3b2-2c03a56baa79 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:04.467640 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1143/2298] 10.222.0.62 () {54 vars in 921 bytes} [Fri Sep 9 12:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:11:04.472598 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86e56b-327a-4c19-8643-de05162ad1c4 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:04.472887 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86e56b-327a-4c19-8643-de05162ad1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:04.473163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86e56b-327a-4c19-8643-de05162ad1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:04.492647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9f86e56b-327a-4c19-8643-de05162ad1c4 None None] MFA Rules not processed for user `1b30e80c421245b5a1ed5849b28ea90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:11:04.493846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9f86e56b-327a-4c19-8643-de05162ad1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:04.504025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1156/2299] 10.222.0.62 () {58 vars in 1018 bytes} [Fri Sep 9 12:11:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:11:04.515075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85f0e3-c3ac-49d3-a3b2-2c03a56baa79 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:11:04.526172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85f0e3-c3ac-49d3-a3b2-2c03a56baa79 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:11:04.529874 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902acff2-3220-4702-93fb-38e2f1692590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:11:04.530496 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902acff2-3220-4702-93fb-38e2f1692590 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:04.530775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902acff2-3220-4702-93fb-38e2f1692590 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:04.531048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902acff2-3220-4702-93fb-38e2f1692590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:04.540958 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902acff2-3220-4702-93fb-38e2f1692590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:11:04.542427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902acff2-3220-4702-93fb-38e2f1692590 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:11:04.553378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1144/2300] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:11:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:11:07.498732 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040d164-2525-47ce-a105-498660ca9586 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:07.499236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040d164-2525-47ce-a105-498660ca9586 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:07.499342 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040d164-2525-47ce-a105-498660ca9586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:07.518828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-3040d164-2525-47ce-a105-498660ca9586 None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:11:07.521120 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3040d164-2525-47ce-a105-498660ca9586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:07.528997 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1157/2301] 10.222.0.62 () {58 vars in 1009 bytes} [Fri Sep 9 12:11:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:11:07.541909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902acff2-3220-4702-93fb-38e2f1692590 service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:11:07.553883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902acff2-3220-4702-93fb-38e2f1692590 service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:11:07.559662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db82ce-0ca9-4a91-ba78-88126620c6c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:11:07.560426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db82ce-0ca9-4a91-ba78-88126620c6c8 service swift] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:07.560805 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db82ce-0ca9-4a91-ba78-88126620c6c8 service swift] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:07.561176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db82ce-0ca9-4a91-ba78-88126620c6c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:07.569940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db82ce-0ca9-4a91-ba78-88126620c6c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:11:07.571654 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db82ce-0ca9-4a91-ba78-88126620c6c8 service swift] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:11:07.580045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1145/2302] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:11:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:11:08.700187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040d164-2525-47ce-a105-498660ca9586 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:11:08.710325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040d164-2525-47ce-a105-498660ca9586 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:11:08.713970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f93fd-2341-4134-8163-bc92e7a36ebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:11:08.714626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f93fd-2341-4134-8163-bc92e7a36ebf service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:08.714902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f93fd-2341-4134-8163-bc92e7a36ebf service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:08.715172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f93fd-2341-4134-8163-bc92e7a36ebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:08.717009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b03f93fd-2341-4134-8163-bc92e7a36ebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:08.731539 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f93fd-2341-4134-8163-bc92e7a36ebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:11:08.732971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f93fd-2341-4134-8163-bc92e7a36ebf service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:11:08.744644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1158/2303] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:11:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:11:09.936987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db82ce-0ca9-4a91-ba78-88126620c6c8 service swift] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:11:09.938646 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b0db82ce-0ca9-4a91-ba78-88126620c6c8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:09.952747 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db82ce-0ca9-4a91-ba78-88126620c6c8 service swift] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:11:09.956649 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c0a06-6b0f-404c-a5c9-40dd8c166cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:11:09.957261 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c0a06-6b0f-404c-a5c9-40dd8c166cce service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:09.957541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c0a06-6b0f-404c-a5c9-40dd8c166cce service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:09.957810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c0a06-6b0f-404c-a5c9-40dd8c166cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:09.959699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-382c0a06-6b0f-404c-a5c9-40dd8c166cce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:09.974096 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c0a06-6b0f-404c-a5c9-40dd8c166cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:11:09.975602 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c0a06-6b0f-404c-a5c9-40dd8c166cce service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:11:09.988018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1146/2304] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:11:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:11:18.739369 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=49 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:11:19.982685 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=76 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:11:23.182190 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f93fd-2341-4134-8163-bc92e7a36ebf service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:11:23.183697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b03f93fd-2341-4134-8163-bc92e7a36ebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:23.253006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f93fd-2341-4134-8163-bc92e7a36ebf service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:11:23.257416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c70fd-afae-4655-8041-0f529ac7fc8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:11:23.258107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c70fd-afae-4655-8041-0f529ac7fc8e service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:23.258518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c70fd-afae-4655-8041-0f529ac7fc8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:23.258885 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c70fd-afae-4655-8041-0f529ac7fc8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:23.260943 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-520c70fd-afae-4655-8041-0f529ac7fc8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:23.324601 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c70fd-afae-4655-8041-0f529ac7fc8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:11:23.326036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c70fd-afae-4655-8041-0f529ac7fc8e service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:11:23.336470 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1159/2305] 10.222.0.62 () {60 vars in 1396 bytes} [Fri Sep 9 12:11:23 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:11:29.274844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c0a06-6b0f-404c-a5c9-40dd8c166cce service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:11:29.276317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-382c0a06-6b0f-404c-a5c9-40dd8c166cce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:29.290344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c0a06-6b0f-404c-a5c9-40dd8c166cce service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:11:29.295829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c80b5-76cd-4e16-976b-da4c6022a014 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:11:29.296422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c80b5-76cd-4e16-976b-da4c6022a014 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:29.297335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c80b5-76cd-4e16-976b-da4c6022a014 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:29.297604 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c80b5-76cd-4e16-976b-da4c6022a014 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:29.299287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-dc6c80b5-76cd-4e16-976b-da4c6022a014 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:29.356702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c80b5-76cd-4e16-976b-da4c6022a014 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:11:29.358333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c80b5-76cd-4e16-976b-da4c6022a014 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:11:29.370292 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1147/2306] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:11:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:11:33.331678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=48 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:11:35.418304 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623f42e-ff92-4336-9863-0f6fb49e15c8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:35.418877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623f42e-ff92-4336-9863-0f6fb49e15c8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:35.418877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623f42e-ff92-4336-9863-0f6fb49e15c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:35.438723 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7623f42e-ff92-4336-9863-0f6fb49e15c8 None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:11:35.439941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7623f42e-ff92-4336-9863-0f6fb49e15c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:35.447179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1160/2307] 10.222.0.62 () {58 vars in 967 bytes} [Fri Sep 9 12:11:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:11:35.460643 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c80b5-76cd-4e16-976b-da4c6022a014 service nova] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:11:35.472929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c80b5-76cd-4e16-976b-da4c6022a014 service nova] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:11:35.477960 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0e851-c54e-4b4e-9ea0-a75ee345e0a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:11:35.478592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0e851-c54e-4b4e-9ea0-a75ee345e0a3 service swift] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:35.478872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0e851-c54e-4b4e-9ea0-a75ee345e0a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:35.479143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0e851-c54e-4b4e-9ea0-a75ee345e0a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:35.487366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0e851-c54e-4b4e-9ea0-a75ee345e0a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:11:35.488952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0e851-c54e-4b4e-9ea0-a75ee345e0a3 service swift] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:11:35.497361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1148/2308] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:11:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:11:45.433919 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:11:45.493817 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=34 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:11:50.405565 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623f42e-ff92-4336-9863-0f6fb49e15c8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:11:50.414864 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623f42e-ff92-4336-9863-0f6fb49e15c8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:11:50.418417 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe32e9-ecc4-4b8e-be9c-b50a33d33297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:11:50.419011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe32e9-ecc4-4b8e-be9c-b50a33d33297 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:50.419289 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe32e9-ecc4-4b8e-be9c-b50a33d33297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:50.419559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe32e9-ecc4-4b8e-be9c-b50a33d33297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:50.421270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fdbe32e9-ecc4-4b8e-be9c-b50a33d33297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:50.430707 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe32e9-ecc4-4b8e-be9c-b50a33d33297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:11:50.432112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe32e9-ecc4-4b8e-be9c-b50a33d33297 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:11:50.440826 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1161/2309] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:11:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:11:52.356118 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990941d4-cbec-485b-8689-da052534ef10 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:52.356627 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990941d4-cbec-485b-8689-da052534ef10 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:52.356769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990941d4-cbec-485b-8689-da052534ef10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:52.377950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-990941d4-cbec-485b-8689-da052534ef10 None None] MFA Rules not processed for user `9c0f1bfeb1464f87a7f3ea4add940861`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:11:52.379124 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-990941d4-cbec-485b-8689-da052534ef10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:52.386135 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1149/2310] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:11:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:11:58.702349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe32e9-ecc4-4b8e-be9c-b50a33d33297 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:11:58.717370 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe32e9-ecc4-4b8e-be9c-b50a33d33297 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:11:58.726466 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56358065-af7c-429c-88bc-73d7387dcccd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:11:58.727645 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56358065-af7c-429c-88bc-73d7387dcccd service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:58.727930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56358065-af7c-429c-88bc-73d7387dcccd service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:58.728199 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56358065-af7c-429c-88bc-73d7387dcccd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:58.732466 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-56358065-af7c-429c-88bc-73d7387dcccd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:58.746966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56358065-af7c-429c-88bc-73d7387dcccd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:11:58.748434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56358065-af7c-429c-88bc-73d7387dcccd service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:11:58.762021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1162/2311] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:11:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:02.370895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:08.757185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=12 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:12.754802 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990941d4-cbec-485b-8689-da052534ef10 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:12.767479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990941d4-cbec-485b-8689-da052534ef10 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:12.772884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03665b15-d6be-4d43-9b06-fc997d81c43e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:12.773471 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03665b15-d6be-4d43-9b06-fc997d81c43e service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:12.773752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03665b15-d6be-4d43-9b06-fc997d81c43e service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:12.774026 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03665b15-d6be-4d43-9b06-fc997d81c43e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:12.775787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-03665b15-d6be-4d43-9b06-fc997d81c43e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:12.835573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03665b15-d6be-4d43-9b06-fc997d81c43e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:12.838190 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03665b15-d6be-4d43-9b06-fc997d81c43e service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:12.849245 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1150/2312] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:22.843900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=28 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:27.804919 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56358065-af7c-429c-88bc-73d7387dcccd service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:27.806463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-56358065-af7c-429c-88bc-73d7387dcccd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:27.820090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56358065-af7c-429c-88bc-73d7387dcccd service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:27.825931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6381a7b-ed74-4623-8fad-100f45fbd5af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:27.826571 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6381a7b-ed74-4623-8fad-100f45fbd5af service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:27.826843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6381a7b-ed74-4623-8fad-100f45fbd5af service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:27.827109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6381a7b-ed74-4623-8fad-100f45fbd5af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:27.828912 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d6381a7b-ed74-4623-8fad-100f45fbd5af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:27.840804 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6381a7b-ed74-4623-8fad-100f45fbd5af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:27.842442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6381a7b-ed74-4623-8fad-100f45fbd5af service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:27.853141 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1163/2313] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:37.849028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:40.576966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f406f3-185e-47ef-b35d-8fb1c11471d2 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.577465 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f406f3-185e-47ef-b35d-8fb1c11471d2 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.577576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f406f3-185e-47ef-b35d-8fb1c11471d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.597167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a6f406f3-185e-47ef-b35d-8fb1c11471d2 None None] MFA Rules not processed for user `16c99b5db9794dff83ea1b67fd238655`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:40.598573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a6f406f3-185e-47ef-b35d-8fb1c11471d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:40.605406 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1151/2314] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:12:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:40.909324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6381a7b-ed74-4623-8fad-100f45fbd5af service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:40.918849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6381a7b-ed74-4623-8fad-100f45fbd5af service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:40.922480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39eb250-4b11-4cd4-97bb-92cb02871095 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:40.923077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39eb250-4b11-4cd4-97bb-92cb02871095 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.923357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39eb250-4b11-4cd4-97bb-92cb02871095 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.923629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39eb250-4b11-4cd4-97bb-92cb02871095 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.925877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c39eb250-4b11-4cd4-97bb-92cb02871095 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:40.935685 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39eb250-4b11-4cd4-97bb-92cb02871095 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:40.937086 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39eb250-4b11-4cd4-97bb-92cb02871095 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:40.945635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1164/2315] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:41.267979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f406f3-185e-47ef-b35d-8fb1c11471d2 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:41.269440 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a6f406f3-185e-47ef-b35d-8fb1c11471d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:41.342273 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f406f3-185e-47ef-b35d-8fb1c11471d2 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:41.347577 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee9d59-3669-4698-b689-8332dfcca9af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:41.348176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee9d59-3669-4698-b689-8332dfcca9af service placement] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:41.348453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee9d59-3669-4698-b689-8332dfcca9af service placement] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:41.348720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee9d59-3669-4698-b689-8332dfcca9af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:41.350492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b5ee9d59-3669-4698-b689-8332dfcca9af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:41.361775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee9d59-3669-4698-b689-8332dfcca9af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:41.363374 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee9d59-3669-4698-b689-8332dfcca9af service placement] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:41.373435 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1152/2316] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:50.942143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:51.369073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=35 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:53.128201 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620df253-1b27-4941-b68e-e6324df70177 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:53.129078 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620df253-1b27-4941-b68e-e6324df70177 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:53.129359 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620df253-1b27-4941-b68e-e6324df70177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:53.161168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-620df253-1b27-4941-b68e-e6324df70177 None None] MFA Rules not processed for user `9676597a1b884138bf5435f3b6cd63c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:53.162676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-620df253-1b27-4941-b68e-e6324df70177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:53.173934 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1165/2317] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:12:53 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:53.288804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee9d59-3669-4698-b689-8332dfcca9af service placement] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:53.303111 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee9d59-3669-4698-b689-8332dfcca9af service placement] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:53.308758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c302b-f08d-43f4-ba54-63cddd8781f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:53.309668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c302b-f08d-43f4-ba54-63cddd8781f4 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:53.309945 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c302b-f08d-43f4-ba54-63cddd8781f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:53.310244 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c302b-f08d-43f4-ba54-63cddd8781f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:53.312045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-ca7c302b-f08d-43f4-ba54-63cddd8781f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:53.326497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c302b-f08d-43f4-ba54-63cddd8781f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:53.328107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c302b-f08d-43f4-ba54-63cddd8781f4 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:53.340909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1153/2318] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:53.513984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620df253-1b27-4941-b68e-e6324df70177 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:53.524906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620df253-1b27-4941-b68e-e6324df70177 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:53.529859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44a9be-dee9-4b3d-b923-bc5adf32fd20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:53.530489 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44a9be-dee9-4b3d-b923-bc5adf32fd20 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:53.530982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44a9be-dee9-4b3d-b923-bc5adf32fd20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:53.531421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44a9be-dee9-4b3d-b923-bc5adf32fd20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:53.537365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0a44a9be-dee9-4b3d-b923-bc5adf32fd20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:53.548760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44a9be-dee9-4b3d-b923-bc5adf32fd20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:53.550217 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44a9be-dee9-4b3d-b923-bc5adf32fd20 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:53.558966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1166/2319] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:03.335426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=6 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:03.554543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=9 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:11.597979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c302b-f08d-43f4-ba54-63cddd8781f4 service nova] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:11.611819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c302b-f08d-43f4-ba54-63cddd8781f4 service nova] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:11.619900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad21731-52e3-4d74-956c-26e9284ed286 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:11.620600 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad21731-52e3-4d74-956c-26e9284ed286 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:11.620953 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad21731-52e3-4d74-956c-26e9284ed286 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:11.621258 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad21731-52e3-4d74-956c-26e9284ed286 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:11.623849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2ad21731-52e3-4d74-956c-26e9284ed286 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:11.640329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad21731-52e3-4d74-956c-26e9284ed286 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:11.642839 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad21731-52e3-4d74-956c-26e9284ed286 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:11.654904 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1154/2320] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:21.649698 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=6 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:22.708213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44a9be-dee9-4b3d-b923-bc5adf32fd20 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:22.717567 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44a9be-dee9-4b3d-b923-bc5adf32fd20 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:22.721400 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb807a-da77-44fb-9701-d680fc42bc09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:22.722002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb807a-da77-44fb-9701-d680fc42bc09 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:22.722311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb807a-da77-44fb-9701-d680fc42bc09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:22.722580 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb807a-da77-44fb-9701-d680fc42bc09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:22.724326 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-65eb807a-da77-44fb-9701-d680fc42bc09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:22.790327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb807a-da77-44fb-9701-d680fc42bc09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:22.791772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb807a-da77-44fb-9701-d680fc42bc09 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:22.802050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1167/2321] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:25.621471 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad21731-52e3-4d74-956c-26e9284ed286 service nova] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:25.622978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2ad21731-52e3-4d74-956c-26e9284ed286 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:25.640433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad21731-52e3-4d74-956c-26e9284ed286 service nova] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:25.646169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21fae1-05e5-422f-b7f4-ef35dc49bc30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:25.647084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21fae1-05e5-422f-b7f4-ef35dc49bc30 service placement] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:25.647304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21fae1-05e5-422f-b7f4-ef35dc49bc30 service placement] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:25.647571 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21fae1-05e5-422f-b7f4-ef35dc49bc30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:25.649412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-fb21fae1-05e5-422f-b7f4-ef35dc49bc30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:25.665255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21fae1-05e5-422f-b7f4-ef35dc49bc30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:25.666900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21fae1-05e5-422f-b7f4-ef35dc49bc30 service placement] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:25.679592 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1155/2322] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:25.700766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb807a-da77-44fb-9701-d680fc42bc09 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:25.702779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-65eb807a-da77-44fb-9701-d680fc42bc09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:25.718507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb807a-da77-44fb-9701-d680fc42bc09 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:25.725248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d99e6-28f4-4d78-9647-0b7e85ef1a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:25.726634 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d99e6-28f4-4d78-9647-0b7e85ef1a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:25.727003 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d99e6-28f4-4d78-9647-0b7e85ef1a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:25.727623 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d99e6-28f4-4d78-9647-0b7e85ef1a28 admin admin] PATH_INFO: `/v3/users/9c0f1bfeb1464f87a7f3ea4add940861` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:25.729258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d99e6-28f4-4d78-9647-0b7e85ef1a28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c0f1bfeb1464f87a7f3ea4add940861)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:25.731694 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d99e6-28f4-4d78-9647-0b7e85ef1a28 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:25.785669 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f17d99e6-28f4-4d78-9647-0b7e85ef1a28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c0f1bfeb1464f87a7f3ea4add940861'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:25.786345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f17d99e6-28f4-4d78-9647-0b7e85ef1a28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c0f1bfeb1464f87a7f3ea4add940861'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:25.793188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f17d99e6-28f4-4d78-9647-0b7e85ef1a28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c0f1bfeb1464f87a7f3ea4add940861'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:25.800946 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f17d99e6-28f4-4d78-9647-0b7e85ef1a28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c0f1bfeb1464f87a7f3ea4add940861'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:25.814010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f17d99e6-28f4-4d78-9647-0b7e85ef1a28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c0f1bfeb1464f87a7f3ea4add940861'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:25.819889 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1168/2323] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:25 2022] DELETE /identity/v3/users/9c0f1bfeb1464f87a7f3ea4add940861 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:26.005073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21fae1-05e5-422f-b7f4-ef35dc49bc30 service placement] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.023002 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21fae1-05e5-422f-b7f4-ef35dc49bc30 service placement] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.081162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa25128-a680-42b4-aca2-568ef9c3d166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.081789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa25128-a680-42b4-aca2-568ef9c3d166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.082068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa25128-a680-42b4-aca2-568ef9c3d166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.082371 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa25128-a680-42b4-aca2-568ef9c3d166 admin admin] PATH_INFO: `/v3/projects/c9860574ff3947e3bc61a76e6883107a` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.083485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa25128-a680-42b4-aca2-568ef9c3d166 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9860574ff3947e3bc61a76e6883107a)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.085390 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa25128-a680-42b4-aca2-568ef9c3d166 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.132110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2fa25128-a680-42b4-aca2-568ef9c3d166 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9860574ff3947e3bc61a76e6883107a', 'request_id': 'req-2fa25128-a680-42b4-aca2-568ef9c3d166'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:26.139685 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2fa25128-a680-42b4-aca2-568ef9c3d166 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9860574ff3947e3bc61a76e6883107a', 'request_id': 'req-2fa25128-a680-42b4-aca2-568ef9c3d166'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:26.147017 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2fa25128-a680-42b4-aca2-568ef9c3d166 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9860574ff3947e3bc61a76e6883107a', 'request_id': 'req-2fa25128-a680-42b4-aca2-568ef9c3d166'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:26.159190 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2fa25128-a680-42b4-aca2-568ef9c3d166 admin admin] The token cache is being invalidate because project c9860574ff3947e3bc61a76e6883107a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:26.159609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2fa25128-a680-42b4-aca2-568ef9c3d166 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:26.160275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2fa25128-a680-42b4-aca2-568ef9c3d166 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:26.162081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1156/2324] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:26 2022] DELETE /identity/v3/projects/c9860574ff3947e3bc61a76e6883107a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:26.166973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31eaaf9-91fb-4581-bb2b-07f2c687c782 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.167251 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31eaaf9-91fb-4581-bb2b-07f2c687c782 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.167558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31eaaf9-91fb-4581-bb2b-07f2c687c782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.189143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f31eaaf9-91fb-4581-bb2b-07f2c687c782 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:26.190300 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f31eaaf9-91fb-4581-bb2b-07f2c687c782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:26.251133 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1169/2325] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:26.259468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe24c3-cdb1-4522-a831-a98829ada22d None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.259755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe24c3-cdb1-4522-a831-a98829ada22d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.260030 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe24c3-cdb1-4522-a831-a98829ada22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.282043 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-68fe24c3-cdb1-4522-a831-a98829ada22d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:26.283313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-68fe24c3-cdb1-4522-a831-a98829ada22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:26.291165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1157/2326] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:26.295393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31eaaf9-91fb-4581-bb2b-07f2c687c782 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.305019 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31eaaf9-91fb-4581-bb2b-07f2c687c782 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.309806 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b82f4-7e9e-4255-bef4-ba0cbe361150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.310443 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b82f4-7e9e-4255-bef4-ba0cbe361150 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.310713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b82f4-7e9e-4255-bef4-ba0cbe361150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.311005 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b82f4-7e9e-4255-bef4-ba0cbe361150 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.311655 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b82f4-7e9e-4255-bef4-ba0cbe361150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:26.311962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b82f4-7e9e-4255-bef4-ba0cbe361150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.313524 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b82f4-7e9e-4255-bef4-ba0cbe361150 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.315669 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:26.327355 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1170/2327] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:13:26.332243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe24c3-cdb1-4522-a831-a98829ada22d None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.341736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe24c3-cdb1-4522-a831-a98829ada22d None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.346433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bf669-a32c-44d4-8213-5a95aa9045ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.347038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bf669-a32c-44d4-8213-5a95aa9045ff admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.347341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bf669-a32c-44d4-8213-5a95aa9045ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.347602 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bf669-a32c-44d4-8213-5a95aa9045ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.348253 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bf669-a32c-44d4-8213-5a95aa9045ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.350148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804bf669-a32c-44d4-8213-5a95aa9045ff admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.378172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1158/2328] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:26 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:26.382569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b82f4-7e9e-4255-bef4-ba0cbe361150 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.393662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b82f4-7e9e-4255-bef4-ba0cbe361150 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.450292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd81016-ab4d-40cc-bf00-32cf27801a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.451185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd81016-ab4d-40cc-bf00-32cf27801a09 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.451464 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd81016-ab4d-40cc-bf00-32cf27801a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.451741 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd81016-ab4d-40cc-bf00-32cf27801a09 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.452373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd81016-ab4d-40cc-bf00-32cf27801a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.454356 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd81016-ab4d-40cc-bf00-32cf27801a09 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.479033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1171/2329] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:26 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:26.483716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bf669-a32c-44d4-8213-5a95aa9045ff admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.494459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bf669-a32c-44d4-8213-5a95aa9045ff admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.499109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60bdf8-e03a-4903-b047-5fa09d15283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.499711 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60bdf8-e03a-4903-b047-5fa09d15283a admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.499986 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60bdf8-e03a-4903-b047-5fa09d15283a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.500253 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60bdf8-e03a-4903-b047-5fa09d15283a admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.500894 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60bdf8-e03a-4903-b047-5fa09d15283a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.503001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60bdf8-e03a-4903-b047-5fa09d15283a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.513395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1159/2330] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:26.518464 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd81016-ab4d-40cc-bf00-32cf27801a09 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.528829 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd81016-ab4d-40cc-bf00-32cf27801a09 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.533793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e02d8-516a-4858-9175-3ee009c2a3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.534456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e02d8-516a-4858-9175-3ee009c2a3fb admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.534781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e02d8-516a-4858-9175-3ee009c2a3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.534986 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e02d8-516a-4858-9175-3ee009c2a3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.535703 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e02d8-516a-4858-9175-3ee009c2a3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.537234 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e02d8-516a-4858-9175-3ee009c2a3fb admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.548559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1172/2331] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:26.552814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60bdf8-e03a-4903-b047-5fa09d15283a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.563126 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60bdf8-e03a-4903-b047-5fa09d15283a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.568613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e3e36-e77d-428e-8fa6-d4a2600b50e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.569187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e3e36-e77d-428e-8fa6-d4a2600b50e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.569460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e3e36-e77d-428e-8fa6-d4a2600b50e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.569729 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e3e36-e77d-428e-8fa6-d4a2600b50e2 admin admin] PATH_INFO: `/v3/projects/ddc4c771b50a4d42ad1689981b217d33/users/2e84b89a71e646f38915d580fe7edc51/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.585929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e3e36-e77d-428e-8fa6-d4a2600b50e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddc4c771b50a4d42ad1689981b217d33, user_id=2e84b89a71e646f38915d580fe7edc51, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.588687 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e3e36-e77d-428e-8fa6-d4a2600b50e2 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.604097 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1160/2332] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:26 2022] PUT /identity/v3/projects/ddc4c771b50a4d42ad1689981b217d33/users/2e84b89a71e646f38915d580fe7edc51/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:26.609790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3feb997-6637-4bf4-a205-5d107b85539a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.610074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3feb997-6637-4bf4-a205-5d107b85539a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.610380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3feb997-6637-4bf4-a205-5d107b85539a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.632738 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f3feb997-6637-4bf4-a205-5d107b85539a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:26.633971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f3feb997-6637-4bf4-a205-5d107b85539a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:26.697065 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1173/2333] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:26.707236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e3e36-e77d-428e-8fa6-d4a2600b50e2 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.709728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-104e3e36-e77d-428e-8fa6-d4a2600b50e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:26.729376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:26.797643 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e3e36-e77d-428e-8fa6-d4a2600b50e2 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.802227 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f9d0a-459e-44c2-92fb-8a0dc28549fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.803344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f9d0a-459e-44c2-92fb-8a0dc28549fc service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.803344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f9d0a-459e-44c2-92fb-8a0dc28549fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.803567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f9d0a-459e-44c2-92fb-8a0dc28549fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.814027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f9d0a-459e-44c2-92fb-8a0dc28549fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.816109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f9d0a-459e-44c2-92fb-8a0dc28549fc service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.827108 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1161/2334] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:27.484303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef9d39-9acd-4ed9-a235-f52aeae943b8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.484918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef9d39-9acd-4ed9-a235-f52aeae943b8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.484918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef9d39-9acd-4ed9-a235-f52aeae943b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.514687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a7ef9d39-9acd-4ed9-a235-f52aeae943b8 None None] MFA Rules not processed for user `2e84b89a71e646f38915d580fe7edc51`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:27.515908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a7ef9d39-9acd-4ed9-a235-f52aeae943b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:27.585901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1174/2335] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:27 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:27.590543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f9d0a-459e-44c2-92fb-8a0dc28549fc service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:27.600591 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f9d0a-459e-44c2-92fb-8a0dc28549fc service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:27.605638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6ddf8-2be5-4b88-ac65-acdea2a185e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:27.606344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6ddf8-2be5-4b88-ac65-acdea2a185e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.606625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6ddf8-2be5-4b88-ac65-acdea2a185e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.606897 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6ddf8-2be5-4b88-ac65-acdea2a185e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.607556 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6ddf8-2be5-4b88-ac65-acdea2a185e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:27.609538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6ddf8-2be5-4b88-ac65-acdea2a185e0 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:27.645714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1162/2336] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:27 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:27.650475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef9d39-9acd-4ed9-a235-f52aeae943b8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:27.657414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:27.664748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef9d39-9acd-4ed9-a235-f52aeae943b8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:27.728132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd818985-1b47-40bd-8b18-3c950807f274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:27.728755 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd818985-1b47-40bd-8b18-3c950807f274 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.729041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd818985-1b47-40bd-8b18-3c950807f274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.729305 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd818985-1b47-40bd-8b18-3c950807f274 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.729958 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd818985-1b47-40bd-8b18-3c950807f274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:27.731921 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd818985-1b47-40bd-8b18-3c950807f274 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:27.758567 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1175/2337] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:27 2022] POST /identity/v3/users => generated 309 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:27.762752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6ddf8-2be5-4b88-ac65-acdea2a185e0 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:27.773382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6ddf8-2be5-4b88-ac65-acdea2a185e0 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:27.778601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3af63-7f48-4f39-8e85-867ec0cc5a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:27.779452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3af63-7f48-4f39-8e85-867ec0cc5a87 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.780153 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3af63-7f48-4f39-8e85-867ec0cc5a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.780429 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3af63-7f48-4f39-8e85-867ec0cc5a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.781087 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3af63-7f48-4f39-8e85-867ec0cc5a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:27.783236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3af63-7f48-4f39-8e85-867ec0cc5a87 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:27.794304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1163/2338] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:27.799325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd818985-1b47-40bd-8b18-3c950807f274 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:27.809676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd818985-1b47-40bd-8b18-3c950807f274 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:27.814549 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1dd9a-59e9-429f-af8c-065d639e4369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:27.815211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1dd9a-59e9-429f-af8c-065d639e4369 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.815492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1dd9a-59e9-429f-af8c-065d639e4369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.815768 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1dd9a-59e9-429f-af8c-065d639e4369 admin admin] PATH_INFO: `/v3/projects/3a377c53e0e44b719ef52c4a46609170/users/71ebc9ead209411f9754e1d686a9d858/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.832522 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1dd9a-59e9-429f-af8c-065d639e4369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a377c53e0e44b719ef52c4a46609170, user_id=71ebc9ead209411f9754e1d686a9d858, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:27.835232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1dd9a-59e9-429f-af8c-065d639e4369 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:27.849196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1176/2339] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:27 2022] PUT /identity/v3/projects/3a377c53e0e44b719ef52c4a46609170/users/71ebc9ead209411f9754e1d686a9d858/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:27.853419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3af63-7f48-4f39-8e85-867ec0cc5a87 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:27.864291 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3af63-7f48-4f39-8e85-867ec0cc5a87 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:27.928237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1efe66-d25a-4dfc-ae58-c9d5d72d7e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:27.928845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1efe66-d25a-4dfc-ae58-c9d5d72d7e77 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.929124 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1efe66-d25a-4dfc-ae58-c9d5d72d7e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.929384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1efe66-d25a-4dfc-ae58-c9d5d72d7e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.930018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1efe66-d25a-4dfc-ae58-c9d5d72d7e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:27.931646 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1efe66-d25a-4dfc-ae58-c9d5d72d7e77 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:27.941438 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1164/2340] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:27.945826 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1dd9a-59e9-429f-af8c-065d639e4369 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:27.959564 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1dd9a-59e9-429f-af8c-065d639e4369 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:27.965949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dcc6a-42a1-4c5f-8e2e-9b38295d15ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:27.966647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dcc6a-42a1-4c5f-8e2e-9b38295d15ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.966920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dcc6a-42a1-4c5f-8e2e-9b38295d15ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.967188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dcc6a-42a1-4c5f-8e2e-9b38295d15ec admin admin] PATH_INFO: `/v3/projects/3a377c53e0e44b719ef52c4a46609170/users/71ebc9ead209411f9754e1d686a9d858/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.969079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77dcc6a-42a1-4c5f-8e2e-9b38295d15ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a377c53e0e44b719ef52c4a46609170, user_id=71ebc9ead209411f9754e1d686a9d858, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:27.972590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77dcc6a-42a1-4c5f-8e2e-9b38295d15ec admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:27.985087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1177/2341] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:27 2022] PUT /identity/v3/projects/3a377c53e0e44b719ef52c4a46609170/users/71ebc9ead209411f9754e1d686a9d858/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:28.757335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbadc42-42e5-4b2f-992f-08ef2f272808 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:28.757678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbadc42-42e5-4b2f-992f-08ef2f272808 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:28.757987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbadc42-42e5-4b2f-992f-08ef2f272808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:28.791831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-1dbadc42-42e5-4b2f-992f-08ef2f272808 None None] MFA Rules not processed for user `71ebc9ead209411f9754e1d686a9d858`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:28.793596 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-1dbadc42-42e5-4b2f-992f-08ef2f272808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:28.886066 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1165/2342] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:28 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:30.323231 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dcc6a-42a1-4c5f-8e2e-9b38295d15ec admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:30.390148 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dcc6a-42a1-4c5f-8e2e-9b38295d15ec admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:30.394704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8ac58-c91e-4577-8573-28ac1beba035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:30.395328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8ac58-c91e-4577-8573-28ac1beba035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:30.395603 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8ac58-c91e-4577-8573-28ac1beba035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:30.395868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8ac58-c91e-4577-8573-28ac1beba035 admin admin] PATH_INFO: `/v3/users/2e84b89a71e646f38915d580fe7edc51` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:30.396972 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8ac58-c91e-4577-8573-28ac1beba035 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e84b89a71e646f38915d580fe7edc51)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:30.398910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8ac58-c91e-4577-8573-28ac1beba035 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:30.446547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5bd8ac58-c91e-4577-8573-28ac1beba035 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e84b89a71e646f38915d580fe7edc51'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:30.447113 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5bd8ac58-c91e-4577-8573-28ac1beba035 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e84b89a71e646f38915d580fe7edc51'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:30.453918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5bd8ac58-c91e-4577-8573-28ac1beba035 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e84b89a71e646f38915d580fe7edc51'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:30.455621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:30.461838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5bd8ac58-c91e-4577-8573-28ac1beba035 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e84b89a71e646f38915d580fe7edc51'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:30.474996 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5bd8ac58-c91e-4577-8573-28ac1beba035 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e84b89a71e646f38915d580fe7edc51'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:30.480961 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1178/2343] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:30 2022] DELETE /identity/v3/users/2e84b89a71e646f38915d580fe7edc51 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:30.667243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbadc42-42e5-4b2f-992f-08ef2f272808 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:30.678898 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:30.680390 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbadc42-42e5-4b2f-992f-08ef2f272808 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:30.754419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a285d7-a165-497d-9cc2-b45a4755da27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:30.755067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a285d7-a165-497d-9cc2-b45a4755da27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:30.755345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a285d7-a165-497d-9cc2-b45a4755da27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:30.755619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a285d7-a165-497d-9cc2-b45a4755da27 admin admin] PATH_INFO: `/v3/projects/ddc4c771b50a4d42ad1689981b217d33` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:30.756766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a285d7-a165-497d-9cc2-b45a4755da27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddc4c771b50a4d42ad1689981b217d33)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:30.758764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a285d7-a165-497d-9cc2-b45a4755da27 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:30.807758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c8a285d7-a165-497d-9cc2-b45a4755da27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddc4c771b50a4d42ad1689981b217d33', 'request_id': 'req-c8a285d7-a165-497d-9cc2-b45a4755da27'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:30.816302 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c8a285d7-a165-497d-9cc2-b45a4755da27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddc4c771b50a4d42ad1689981b217d33', 'request_id': 'req-c8a285d7-a165-497d-9cc2-b45a4755da27'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:30.823218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c8a285d7-a165-497d-9cc2-b45a4755da27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddc4c771b50a4d42ad1689981b217d33', 'request_id': 'req-c8a285d7-a165-497d-9cc2-b45a4755da27'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:30.835806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c8a285d7-a165-497d-9cc2-b45a4755da27 admin admin] The token cache is being invalidate because project ddc4c771b50a4d42ad1689981b217d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:30.836150 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c8a285d7-a165-497d-9cc2-b45a4755da27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:30.836836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c8a285d7-a165-497d-9cc2-b45a4755da27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:30.838977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1166/2344] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:30 2022] DELETE /identity/v3/projects/ddc4c771b50a4d42ad1689981b217d33 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:30.843160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8ac58-c91e-4577-8573-28ac1beba035 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:30.844764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5bd8ac58-c91e-4577-8573-28ac1beba035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:30.857783 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8ac58-c91e-4577-8573-28ac1beba035 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:30.919958 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdff37-9522-477c-896d-dd0ac87768f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:30.920636 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdff37-9522-477c-896d-dd0ac87768f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:30.920912 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdff37-9522-477c-896d-dd0ac87768f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:30.921209 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdff37-9522-477c-896d-dd0ac87768f0 admin admin] PATH_INFO: `/v3/users/71ebc9ead209411f9754e1d686a9d858` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:30.923936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbdff37-9522-477c-896d-dd0ac87768f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71ebc9ead209411f9754e1d686a9d858)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:30.926666 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbdff37-9522-477c-896d-dd0ac87768f0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:30.986220 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfbdff37-9522-477c-896d-dd0ac87768f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71ebc9ead209411f9754e1d686a9d858'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:30.987172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfbdff37-9522-477c-896d-dd0ac87768f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71ebc9ead209411f9754e1d686a9d858'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:30.994784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfbdff37-9522-477c-896d-dd0ac87768f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71ebc9ead209411f9754e1d686a9d858'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:31.003188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfbdff37-9522-477c-896d-dd0ac87768f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71ebc9ead209411f9754e1d686a9d858'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:31.015767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfbdff37-9522-477c-896d-dd0ac87768f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71ebc9ead209411f9754e1d686a9d858'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:31.022592 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1179/2345] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:30 2022] DELETE /identity/v3/users/71ebc9ead209411f9754e1d686a9d858 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:31.249138 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a285d7-a165-497d-9cc2-b45a4755da27 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:31.250163 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-c8a285d7-a165-497d-9cc2-b45a4755da27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:31.276265 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a285d7-a165-497d-9cc2-b45a4755da27 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:31.339865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542b60f-163f-4e48-a414-f70fdef1a053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:31.340508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542b60f-163f-4e48-a414-f70fdef1a053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:31.340785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542b60f-163f-4e48-a414-f70fdef1a053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:31.341054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542b60f-163f-4e48-a414-f70fdef1a053 admin admin] PATH_INFO: `/v3/projects/3a377c53e0e44b719ef52c4a46609170` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:31.342230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542b60f-163f-4e48-a414-f70fdef1a053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a377c53e0e44b719ef52c4a46609170)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:31.344134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542b60f-163f-4e48-a414-f70fdef1a053 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:31.386890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:31.398340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7542b60f-163f-4e48-a414-f70fdef1a053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a377c53e0e44b719ef52c4a46609170', 'request_id': 'req-7542b60f-163f-4e48-a414-f70fdef1a053'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:31.406674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7542b60f-163f-4e48-a414-f70fdef1a053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a377c53e0e44b719ef52c4a46609170', 'request_id': 'req-7542b60f-163f-4e48-a414-f70fdef1a053'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:31.412942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7542b60f-163f-4e48-a414-f70fdef1a053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a377c53e0e44b719ef52c4a46609170', 'request_id': 'req-7542b60f-163f-4e48-a414-f70fdef1a053'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:31.425767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7542b60f-163f-4e48-a414-f70fdef1a053 admin admin] The token cache is being invalidate because project 3a377c53e0e44b719ef52c4a46609170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:31.426054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7542b60f-163f-4e48-a414-f70fdef1a053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:31.426736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7542b60f-163f-4e48-a414-f70fdef1a053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:31.428472 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1167/2346] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:31 2022] DELETE /identity/v3/projects/3a377c53e0e44b719ef52c4a46609170 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:31.433412 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017e23f-d3ee-4c20-9cb1-1a2a9d7b70d7 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:31.433993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017e23f-d3ee-4c20-9cb1-1a2a9d7b70d7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:31.433993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017e23f-d3ee-4c20-9cb1-1a2a9d7b70d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:31.460727 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4017e23f-d3ee-4c20-9cb1-1a2a9d7b70d7 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:31.462093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4017e23f-d3ee-4c20-9cb1-1a2a9d7b70d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:31.517854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:31.535982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1180/2347] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:31.545317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b943c0-ce1b-4327-b405-d15bb6e1f132 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:31.545616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b943c0-ce1b-4327-b405-d15bb6e1f132 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:31.545859 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b943c0-ce1b-4327-b405-d15bb6e1f132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:31.569179 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-c5b943c0-ce1b-4327-b405-d15bb6e1f132 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:31.570567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-c5b943c0-ce1b-4327-b405-d15bb6e1f132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:31.578981 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1168/2348] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:31.583516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017e23f-d3ee-4c20-9cb1-1a2a9d7b70d7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:31.593258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017e23f-d3ee-4c20-9cb1-1a2a9d7b70d7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:31.600089 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b88a5f-6852-46fa-9eea-ed192cd07fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:31.600883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b88a5f-6852-46fa-9eea-ed192cd07fea admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:31.601169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b88a5f-6852-46fa-9eea-ed192cd07fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:31.601438 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b88a5f-6852-46fa-9eea-ed192cd07fea admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:31.602097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b88a5f-6852-46fa-9eea-ed192cd07fea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:31.602447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b88a5f-6852-46fa-9eea-ed192cd07fea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:31.604043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b88a5f-6852-46fa-9eea-ed192cd07fea admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:31.618517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1181/2349] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:13:31.623564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b943c0-ce1b-4327-b405-d15bb6e1f132 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:31.634198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b943c0-ce1b-4327-b405-d15bb6e1f132 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:31.638931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd672e-18ac-40fa-a219-d38aafa26d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:31.639515 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd672e-18ac-40fa-a219-d38aafa26d84 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:31.639788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd672e-18ac-40fa-a219-d38aafa26d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:31.640243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd672e-18ac-40fa-a219-d38aafa26d84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:31.640877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd672e-18ac-40fa-a219-d38aafa26d84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:31.642876 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd672e-18ac-40fa-a219-d38aafa26d84 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:31.671142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1169/2350] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:31 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:31.676034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b88a5f-6852-46fa-9eea-ed192cd07fea admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:31.686764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b88a5f-6852-46fa-9eea-ed192cd07fea admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:31.754212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69d9d6-51f4-4e77-9854-d4de7d740199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:31.754846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69d9d6-51f4-4e77-9854-d4de7d740199 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:31.755153 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69d9d6-51f4-4e77-9854-d4de7d740199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:31.755512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69d9d6-51f4-4e77-9854-d4de7d740199 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:31.756875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69d9d6-51f4-4e77-9854-d4de7d740199 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:31.759646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69d9d6-51f4-4e77-9854-d4de7d740199 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:31.790823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1182/2351] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:31 2022] POST /identity/v3/users => generated 283 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:31.795305 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd672e-18ac-40fa-a219-d38aafa26d84 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:31.805706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd672e-18ac-40fa-a219-d38aafa26d84 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:31.810344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c728d-1788-4e07-8c8a-15a48a51dd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:31.810971 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c728d-1788-4e07-8c8a-15a48a51dd15 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:31.811246 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c728d-1788-4e07-8c8a-15a48a51dd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:31.811514 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c728d-1788-4e07-8c8a-15a48a51dd15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:31.812167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c728d-1788-4e07-8c8a-15a48a51dd15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:31.814273 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c728d-1788-4e07-8c8a-15a48a51dd15 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:31.824855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1170/2352] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:31.829106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69d9d6-51f4-4e77-9854-d4de7d740199 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:31.840370 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69d9d6-51f4-4e77-9854-d4de7d740199 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:31.845225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d4a5f-0195-4387-94af-1d55138d1d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:31.845827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d4a5f-0195-4387-94af-1d55138d1d82 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:31.846097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d4a5f-0195-4387-94af-1d55138d1d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:31.846394 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d4a5f-0195-4387-94af-1d55138d1d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:31.847027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8d4a5f-0195-4387-94af-1d55138d1d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:31.848631 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8d4a5f-0195-4387-94af-1d55138d1d82 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:31.859799 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1183/2353] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:31.864544 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c728d-1788-4e07-8c8a-15a48a51dd15 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:31.874847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c728d-1788-4e07-8c8a-15a48a51dd15 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:31.880893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440de71-a6fb-4dc6-ab6e-379432765a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:31.881577 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440de71-a6fb-4dc6-ab6e-379432765a84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:31.881858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440de71-a6fb-4dc6-ab6e-379432765a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:31.882162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440de71-a6fb-4dc6-ab6e-379432765a84 admin admin] PATH_INFO: `/v3/projects/00383d6b88ec40c7839dc79c05a07e6a/users/ec866a42e6784b64a5219307f4eb975f/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:31.900074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440de71-a6fb-4dc6-ab6e-379432765a84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00383d6b88ec40c7839dc79c05a07e6a, user_id=ec866a42e6784b64a5219307f4eb975f, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:31.902775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440de71-a6fb-4dc6-ab6e-379432765a84 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:31.933686 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1171/2354] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:31 2022] PUT /identity/v3/projects/00383d6b88ec40c7839dc79c05a07e6a/users/ec866a42e6784b64a5219307f4eb975f/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:31.935728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5005b34-5ef4-4194-9137-482400153843 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:31.936051 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5005b34-5ef4-4194-9137-482400153843 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:31.936296 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5005b34-5ef4-4194-9137-482400153843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:31.965688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f5005b34-5ef4-4194-9137-482400153843 None None] MFA Rules not processed for user `ec866a42e6784b64a5219307f4eb975f`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:31.966946 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f5005b34-5ef4-4194-9137-482400153843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:32.035757 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1184/2355] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:32.040423 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440de71-a6fb-4dc6-ab6e-379432765a84 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:32.103780 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440de71-a6fb-4dc6-ab6e-379432765a84 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:32.108937 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ca613-a4e5-4993-86e1-a857939261bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:32.109545 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ca613-a4e5-4993-86e1-a857939261bc admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:32.109832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ca613-a4e5-4993-86e1-a857939261bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:32.110103 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ca613-a4e5-4993-86e1-a857939261bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:32.111343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ca613-a4e5-4993-86e1-a857939261bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:32.113239 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ca613-a4e5-4993-86e1-a857939261bc admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:32.141740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1172/2356] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:32 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:32.146332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5005b34-5ef4-4194-9137-482400153843 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:32.157816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5005b34-5ef4-4194-9137-482400153843 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:32.212899 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:32.233097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e3969-7752-48f7-85d5-278418f40fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:32.233716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e3969-7752-48f7-85d5-278418f40fae admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:32.234000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e3969-7752-48f7-85d5-278418f40fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:32.234303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e3969-7752-48f7-85d5-278418f40fae admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:32.234938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e3969-7752-48f7-85d5-278418f40fae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:32.237410 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e3969-7752-48f7-85d5-278418f40fae admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:32.261813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1185/2357] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:32 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:32.265602 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ca613-a4e5-4993-86e1-a857939261bc admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:32.275908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ca613-a4e5-4993-86e1-a857939261bc admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:32.280537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79355db3-85b6-4d3b-84db-d5b2020179f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:32.281142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79355db3-85b6-4d3b-84db-d5b2020179f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:32.281417 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79355db3-85b6-4d3b-84db-d5b2020179f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:32.281687 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79355db3-85b6-4d3b-84db-d5b2020179f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:32.283106 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79355db3-85b6-4d3b-84db-d5b2020179f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:32.284479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79355db3-85b6-4d3b-84db-d5b2020179f4 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:32.296115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1173/2358] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:32.300797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e3969-7752-48f7-85d5-278418f40fae admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:32.310855 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e3969-7752-48f7-85d5-278418f40fae admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:32.315820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07cd1d-ed86-4572-b718-59cb6d0759a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:32.316659 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07cd1d-ed86-4572-b718-59cb6d0759a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:32.317060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07cd1d-ed86-4572-b718-59cb6d0759a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:32.317335 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07cd1d-ed86-4572-b718-59cb6d0759a5 admin admin] PATH_INFO: `/v3/projects/9176325edda3461cbcd26b64974e5969/users/5540a8f3e083491aa95c8d5ea3716f2a/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:32.333056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07cd1d-ed86-4572-b718-59cb6d0759a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9176325edda3461cbcd26b64974e5969, user_id=5540a8f3e083491aa95c8d5ea3716f2a, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:32.335759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07cd1d-ed86-4572-b718-59cb6d0759a5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:32.350481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1186/2359] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:32 2022] PUT /identity/v3/projects/9176325edda3461cbcd26b64974e5969/users/5540a8f3e083491aa95c8d5ea3716f2a/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:32.354462 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79355db3-85b6-4d3b-84db-d5b2020179f4 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:32.365362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79355db3-85b6-4d3b-84db-d5b2020179f4 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:32.402646 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:32.429581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4248f86-1c3a-464a-b6fc-c86c4f17d030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:32.430224 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4248f86-1c3a-464a-b6fc-c86c4f17d030 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:32.430504 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4248f86-1c3a-464a-b6fc-c86c4f17d030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:32.430780 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4248f86-1c3a-464a-b6fc-c86c4f17d030 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:32.431423 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4248f86-1c3a-464a-b6fc-c86c4f17d030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:32.433016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4248f86-1c3a-464a-b6fc-c86c4f17d030 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:32.443020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1174/2360] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:13:32.447187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07cd1d-ed86-4572-b718-59cb6d0759a5 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:32.457688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07cd1d-ed86-4572-b718-59cb6d0759a5 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:32.462372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2c778-9ef3-439d-b70b-7ea861a76696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:32.463027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2c778-9ef3-439d-b70b-7ea861a76696 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:32.463329 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2c778-9ef3-439d-b70b-7ea861a76696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:32.463605 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2c778-9ef3-439d-b70b-7ea861a76696 admin admin] PATH_INFO: `/v3/projects/9176325edda3461cbcd26b64974e5969/users/5540a8f3e083491aa95c8d5ea3716f2a/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:32.465513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2c778-9ef3-439d-b70b-7ea861a76696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9176325edda3461cbcd26b64974e5969, user_id=5540a8f3e083491aa95c8d5ea3716f2a, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:32.468218 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2c778-9ef3-439d-b70b-7ea861a76696 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:32.479051 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1187/2361] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:32 2022] PUT /identity/v3/projects/9176325edda3461cbcd26b64974e5969/users/5540a8f3e083491aa95c8d5ea3716f2a/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:32.491027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b274200-3098-4beb-ad17-8b125ab1217a None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:32.491312 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b274200-3098-4beb-ad17-8b125ab1217a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:32.491605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b274200-3098-4beb-ad17-8b125ab1217a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:32.520554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-7b274200-3098-4beb-ad17-8b125ab1217a None None] MFA Rules not processed for user `5540a8f3e083491aa95c8d5ea3716f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:32.521691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7b274200-3098-4beb-ad17-8b125ab1217a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:32.592160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf114cd-4508-470b-ab90-e7e901bab56f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:32.592456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf114cd-4508-470b-ab90-e7e901bab56f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:32.592700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf114cd-4508-470b-ab90-e7e901bab56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:32.605167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1175/2362] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:32.617344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b274200-3098-4beb-ad17-8b125ab1217a None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:32.618763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7b274200-3098-4beb-ad17-8b125ab1217a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:32.621242 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-cdf114cd-4508-470b-ab90-e7e901bab56f None None] MFA Rules not processed for user `bfc29d15a7344754aa53d55379e2751f`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:32.622438 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cdf114cd-4508-470b-ab90-e7e901bab56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:32.678348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b274200-3098-4beb-ad17-8b125ab1217a None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:32.680354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1188/2363] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:32.686965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf77b39-7b2d-4fcd-90bd-0938a4bb60e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:32.687861 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf77b39-7b2d-4fcd-90bd-0938a4bb60e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:32.688264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf77b39-7b2d-4fcd-90bd-0938a4bb60e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:32.688665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf77b39-7b2d-4fcd-90bd-0938a4bb60e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:32.699814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf77b39-7b2d-4fcd-90bd-0938a4bb60e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:32.701241 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf77b39-7b2d-4fcd-90bd-0938a4bb60e5 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:32.711990 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1176/2364] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:33.172112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf114cd-4508-470b-ab90-e7e901bab56f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:33.377448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf114cd-4508-470b-ab90-e7e901bab56f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:33.382180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01e15b-ebf3-44e6-be8c-a7b129fb0855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:33.382951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01e15b-ebf3-44e6-be8c-a7b129fb0855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:33.383604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01e15b-ebf3-44e6-be8c-a7b129fb0855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:33.383604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01e15b-ebf3-44e6-be8c-a7b129fb0855 admin admin] PATH_INFO: `/v3/users/ec866a42e6784b64a5219307f4eb975f` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:33.384610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01e15b-ebf3-44e6-be8c-a7b129fb0855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec866a42e6784b64a5219307f4eb975f)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:33.386564 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01e15b-ebf3-44e6-be8c-a7b129fb0855 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:33.440184 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cf01e15b-ebf3-44e6-be8c-a7b129fb0855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec866a42e6784b64a5219307f4eb975f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:33.440880 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cf01e15b-ebf3-44e6-be8c-a7b129fb0855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec866a42e6784b64a5219307f4eb975f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:33.442185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:33.448708 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cf01e15b-ebf3-44e6-be8c-a7b129fb0855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec866a42e6784b64a5219307f4eb975f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:33.457943 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cf01e15b-ebf3-44e6-be8c-a7b129fb0855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec866a42e6784b64a5219307f4eb975f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:33.470837 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cf01e15b-ebf3-44e6-be8c-a7b129fb0855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec866a42e6784b64a5219307f4eb975f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:33.477733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1189/2365] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:33 2022] DELETE /identity/v3/users/ec866a42e6784b64a5219307f4eb975f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:33.482326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5c949-cb3a-4d5e-9517-46d4b51e2d85 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:33.482671 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5c949-cb3a-4d5e-9517-46d4b51e2d85 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:33.482901 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5c949-cb3a-4d5e-9517-46d4b51e2d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:33.503972 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f3f5c949-cb3a-4d5e-9517-46d4b51e2d85 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:33.506507 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f3f5c949-cb3a-4d5e-9517-46d4b51e2d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:33.562742 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:33.572798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1177/2366] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:33.590565 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01e15b-ebf3-44e6-be8c-a7b129fb0855 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:33.592130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cf01e15b-ebf3-44e6-be8c-a7b129fb0855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:33.652711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01e15b-ebf3-44e6-be8c-a7b129fb0855 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:33.656358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2410c0-b88b-44fb-865c-60cb98c0b61f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:33.656962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2410c0-b88b-44fb-865c-60cb98c0b61f service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:33.657244 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2410c0-b88b-44fb-865c-60cb98c0b61f service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:33.657536 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2410c0-b88b-44fb-865c-60cb98c0b61f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:33.666942 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2410c0-b88b-44fb-865c-60cb98c0b61f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:33.668296 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2410c0-b88b-44fb-865c-60cb98c0b61f service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:33.679944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1190/2367] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:33.986736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5c949-cb3a-4d5e-9517-46d4b51e2d85 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:33.988390 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f3f5c949-cb3a-4d5e-9517-46d4b51e2d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:34.001768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5c949-cb3a-4d5e-9517-46d4b51e2d85 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:34.006659 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a3fa9-72af-42b7-929d-690bbf64fa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:34.007306 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a3fa9-72af-42b7-929d-690bbf64fa69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.007585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a3fa9-72af-42b7-929d-690bbf64fa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.007862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a3fa9-72af-42b7-929d-690bbf64fa69 admin admin] PATH_INFO: `/v3/projects/00383d6b88ec40c7839dc79c05a07e6a` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.009000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a3fa9-72af-42b7-929d-690bbf64fa69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00383d6b88ec40c7839dc79c05a07e6a)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:34.010964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a3fa9-72af-42b7-929d-690bbf64fa69 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:34.059779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-925a3fa9-72af-42b7-929d-690bbf64fa69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00383d6b88ec40c7839dc79c05a07e6a', 'request_id': 'req-925a3fa9-72af-42b7-929d-690bbf64fa69'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.068490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-925a3fa9-72af-42b7-929d-690bbf64fa69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00383d6b88ec40c7839dc79c05a07e6a', 'request_id': 'req-925a3fa9-72af-42b7-929d-690bbf64fa69'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.074469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-925a3fa9-72af-42b7-929d-690bbf64fa69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00383d6b88ec40c7839dc79c05a07e6a', 'request_id': 'req-925a3fa9-72af-42b7-929d-690bbf64fa69'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.087236 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-925a3fa9-72af-42b7-929d-690bbf64fa69 admin admin] The token cache is being invalidate because project 00383d6b88ec40c7839dc79c05a07e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:34.087521 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-925a3fa9-72af-42b7-929d-690bbf64fa69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.088261 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-925a3fa9-72af-42b7-929d-690bbf64fa69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.089965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1178/2368] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:33 2022] DELETE /identity/v3/projects/00383d6b88ec40c7839dc79c05a07e6a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:34.094000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2410c0-b88b-44fb-865c-60cb98c0b61f service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:34.104441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2410c0-b88b-44fb-865c-60cb98c0b61f service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:34.164733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08081405-7d0f-4d73-a968-5a92a7ad5f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:34.165358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08081405-7d0f-4d73-a968-5a92a7ad5f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.165629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08081405-7d0f-4d73-a968-5a92a7ad5f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.165892 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08081405-7d0f-4d73-a968-5a92a7ad5f62 admin admin] PATH_INFO: `/v3/users/5540a8f3e083491aa95c8d5ea3716f2a` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.167029 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08081405-7d0f-4d73-a968-5a92a7ad5f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5540a8f3e083491aa95c8d5ea3716f2a)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:34.168914 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08081405-7d0f-4d73-a968-5a92a7ad5f62 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:34.215143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-08081405-7d0f-4d73-a968-5a92a7ad5f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5540a8f3e083491aa95c8d5ea3716f2a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.215778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-08081405-7d0f-4d73-a968-5a92a7ad5f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5540a8f3e083491aa95c8d5ea3716f2a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.222916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-08081405-7d0f-4d73-a968-5a92a7ad5f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5540a8f3e083491aa95c8d5ea3716f2a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.230494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-08081405-7d0f-4d73-a968-5a92a7ad5f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5540a8f3e083491aa95c8d5ea3716f2a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.236183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:34.246342 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-08081405-7d0f-4d73-a968-5a92a7ad5f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5540a8f3e083491aa95c8d5ea3716f2a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.254487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1191/2369] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:34 2022] DELETE /identity/v3/users/5540a8f3e083491aa95c8d5ea3716f2a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:34.579173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a3fa9-72af-42b7-929d-690bbf64fa69 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:34.581152 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-925a3fa9-72af-42b7-929d-690bbf64fa69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:34.607836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a3fa9-72af-42b7-929d-690bbf64fa69 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:34.670415 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4796108-9a0d-4529-ab27-306536ec2365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:34.671187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4796108-9a0d-4529-ab27-306536ec2365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.671624 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4796108-9a0d-4529-ab27-306536ec2365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.671993 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4796108-9a0d-4529-ab27-306536ec2365 admin admin] PATH_INFO: `/v3/projects/9176325edda3461cbcd26b64974e5969` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.673820 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4796108-9a0d-4529-ab27-306536ec2365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9176325edda3461cbcd26b64974e5969)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:34.675868 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4796108-9a0d-4529-ab27-306536ec2365 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:34.725714 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c4796108-9a0d-4529-ab27-306536ec2365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9176325edda3461cbcd26b64974e5969', 'request_id': 'req-c4796108-9a0d-4529-ab27-306536ec2365'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.733223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c4796108-9a0d-4529-ab27-306536ec2365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9176325edda3461cbcd26b64974e5969', 'request_id': 'req-c4796108-9a0d-4529-ab27-306536ec2365'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.739092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c4796108-9a0d-4529-ab27-306536ec2365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9176325edda3461cbcd26b64974e5969', 'request_id': 'req-c4796108-9a0d-4529-ab27-306536ec2365'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.750185 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c4796108-9a0d-4529-ab27-306536ec2365 admin admin] The token cache is being invalidate because project 9176325edda3461cbcd26b64974e5969 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:34.750472 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c4796108-9a0d-4529-ab27-306536ec2365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.751632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c4796108-9a0d-4529-ab27-306536ec2365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.753165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1179/2370] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:34 2022] DELETE /identity/v3/projects/9176325edda3461cbcd26b64974e5969 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:34.758127 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9ca58-e05e-4db4-b015-61da70c38b0b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.758604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9ca58-e05e-4db4-b015-61da70c38b0b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.758697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9ca58-e05e-4db4-b015-61da70c38b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.778282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-38e9ca58-e05e-4db4-b015-61da70c38b0b None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:34.779459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-38e9ca58-e05e-4db4-b015-61da70c38b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:34.837870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1192/2371] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:34.846943 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a2a7a-4dca-4f51-891e-77a1bd54158f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.847230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a2a7a-4dca-4f51-891e-77a1bd54158f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.847509 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a2a7a-4dca-4f51-891e-77a1bd54158f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.868539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-962a2a7a-4dca-4f51-891e-77a1bd54158f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:34.869926 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-962a2a7a-4dca-4f51-891e-77a1bd54158f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:34.886160 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1180/2372] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:34.886385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9ca58-e05e-4db4-b015-61da70c38b0b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:34.893691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9ca58-e05e-4db4-b015-61da70c38b0b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:34.898419 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3868da-9a97-47b2-bf6a-822381af74e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:34.899027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3868da-9a97-47b2-bf6a-822381af74e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.899308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3868da-9a97-47b2-bf6a-822381af74e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.899577 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3868da-9a97-47b2-bf6a-822381af74e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.900226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3868da-9a97-47b2-bf6a-822381af74e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:34.900526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3868da-9a97-47b2-bf6a-822381af74e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:34.902146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3868da-9a97-47b2-bf6a-822381af74e3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:34.914446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1193/2373] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:13:34.919607 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a2a7a-4dca-4f51-891e-77a1bd54158f None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:34.930969 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a2a7a-4dca-4f51-891e-77a1bd54158f None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:34.936281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a84238-040b-4de0-8126-584dcb268301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:34.936894 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a84238-040b-4de0-8126-584dcb268301 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.937195 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a84238-040b-4de0-8126-584dcb268301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.937496 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a84238-040b-4de0-8126-584dcb268301 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.938194 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a84238-040b-4de0-8126-584dcb268301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:34.940696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a84238-040b-4de0-8126-584dcb268301 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:34.945524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:34.972918 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1181/2374] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:34 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:34.974136 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3868da-9a97-47b2-bf6a-822381af74e3 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:34.984186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3868da-9a97-47b2-bf6a-822381af74e3 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:35.041602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27c1a2-17a2-410f-8ddd-1ded2b1fbdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:35.042247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27c1a2-17a2-410f-8ddd-1ded2b1fbdb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:35.042521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27c1a2-17a2-410f-8ddd-1ded2b1fbdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:35.042787 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27c1a2-17a2-410f-8ddd-1ded2b1fbdb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:35.043446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27c1a2-17a2-410f-8ddd-1ded2b1fbdb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:35.045333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27c1a2-17a2-410f-8ddd-1ded2b1fbdb4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:35.068059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1194/2375] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:34 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:35.072247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a84238-040b-4de0-8126-584dcb268301 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:35.082927 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a84238-040b-4de0-8126-584dcb268301 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:35.087776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b38c2b-4634-4949-b6bb-03cbcf230675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:35.088505 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b38c2b-4634-4949-b6bb-03cbcf230675 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:35.088790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b38c2b-4634-4949-b6bb-03cbcf230675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:35.089063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b38c2b-4634-4949-b6bb-03cbcf230675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:35.089919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b38c2b-4634-4949-b6bb-03cbcf230675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:35.091566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b38c2b-4634-4949-b6bb-03cbcf230675 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:35.102200 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1182/2376] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:35.106378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27c1a2-17a2-410f-8ddd-1ded2b1fbdb4 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:35.117203 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27c1a2-17a2-410f-8ddd-1ded2b1fbdb4 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:35.122286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739aab35-42e7-4ac0-b8ea-fc173cc3f6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:35.122827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739aab35-42e7-4ac0-b8ea-fc173cc3f6ca admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:35.123102 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739aab35-42e7-4ac0-b8ea-fc173cc3f6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:35.123370 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739aab35-42e7-4ac0-b8ea-fc173cc3f6ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:35.124042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739aab35-42e7-4ac0-b8ea-fc173cc3f6ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:35.125624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739aab35-42e7-4ac0-b8ea-fc173cc3f6ca admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:35.136737 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1195/2377] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:35.141113 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b38c2b-4634-4949-b6bb-03cbcf230675 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:35.151872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b38c2b-4634-4949-b6bb-03cbcf230675 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:35.156593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e1568-4998-41cc-a1b0-667486339dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:35.157257 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e1568-4998-41cc-a1b0-667486339dde admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:35.157818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e1568-4998-41cc-a1b0-667486339dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:35.158143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e1568-4998-41cc-a1b0-667486339dde admin admin] PATH_INFO: `/v3/projects/6dd9d777bc58457391e321c8b9781248/users/61fd0dabc6e44ac4b0ad31619475d6b2/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:35.173665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e1568-4998-41cc-a1b0-667486339dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd9d777bc58457391e321c8b9781248, user_id=61fd0dabc6e44ac4b0ad31619475d6b2, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:35.176384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e1568-4998-41cc-a1b0-667486339dde admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:35.203762 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1183/2378] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:35 2022] PUT /identity/v3/projects/6dd9d777bc58457391e321c8b9781248/users/61fd0dabc6e44ac4b0ad31619475d6b2/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:35.205856 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc50e16-5321-4147-8dd8-fc6f1e48dd3f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:35.206215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc50e16-5321-4147-8dd8-fc6f1e48dd3f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:35.206472 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc50e16-5321-4147-8dd8-fc6f1e48dd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:35.240017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1fc50e16-5321-4147-8dd8-fc6f1e48dd3f None None] MFA Rules not processed for user `61fd0dabc6e44ac4b0ad31619475d6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:35.241243 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1fc50e16-5321-4147-8dd8-fc6f1e48dd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:35.289984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:35.320998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1196/2379] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:35.325198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e1568-4998-41cc-a1b0-667486339dde admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:35.406963 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e1568-4998-41cc-a1b0-667486339dde admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:35.413158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f71556-f90c-427e-9b66-e0c2014ca47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:35.413935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f71556-f90c-427e-9b66-e0c2014ca47d admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:35.414337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f71556-f90c-427e-9b66-e0c2014ca47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:35.414703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f71556-f90c-427e-9b66-e0c2014ca47d admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:35.415452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f71556-f90c-427e-9b66-e0c2014ca47d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:35.417508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f71556-f90c-427e-9b66-e0c2014ca47d admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:35.451712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1184/2380] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:35 2022] POST /identity/v3/projects => generated 365 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:35.453335 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc50e16-5321-4147-8dd8-fc6f1e48dd3f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:35.464857 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc50e16-5321-4147-8dd8-fc6f1e48dd3f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:35.529857 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0e64f-9295-4bfc-aaca-def7d9782378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:35.530520 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0e64f-9295-4bfc-aaca-def7d9782378 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:35.530906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0e64f-9295-4bfc-aaca-def7d9782378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:35.532031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0e64f-9295-4bfc-aaca-def7d9782378 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:35.532031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0e64f-9295-4bfc-aaca-def7d9782378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:35.533701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0e64f-9295-4bfc-aaca-def7d9782378 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:35.558641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1197/2381] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:35 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:35.560067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f71556-f90c-427e-9b66-e0c2014ca47d admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:35.577258 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f71556-f90c-427e-9b66-e0c2014ca47d admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:35.586087 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed121092-151d-407b-9777-a69b4c1796ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:35.587684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed121092-151d-407b-9777-a69b4c1796ce admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:35.588041 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed121092-151d-407b-9777-a69b4c1796ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:35.588381 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed121092-151d-407b-9777-a69b4c1796ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:35.589108 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed121092-151d-407b-9777-a69b4c1796ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:35.592349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed121092-151d-407b-9777-a69b4c1796ce admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:35.610269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1185/2382] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:35.615728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0e64f-9295-4bfc-aaca-def7d9782378 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:35.627358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0e64f-9295-4bfc-aaca-def7d9782378 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:35.632175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38761b19-79ba-49c0-8649-568d0b5528fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:35.632890 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38761b19-79ba-49c0-8649-568d0b5528fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:35.633100 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38761b19-79ba-49c0-8649-568d0b5528fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:35.633365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38761b19-79ba-49c0-8649-568d0b5528fd admin admin] PATH_INFO: `/v3/projects/050433d7c3d846ac83edebdf8cd14ef1/users/ab5a0a0292f04c0db64f5154de1f262c/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:35.649478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38761b19-79ba-49c0-8649-568d0b5528fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=050433d7c3d846ac83edebdf8cd14ef1, user_id=ab5a0a0292f04c0db64f5154de1f262c, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:35.652124 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38761b19-79ba-49c0-8649-568d0b5528fd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:35.665094 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1198/2383] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:35 2022] PUT /identity/v3/projects/050433d7c3d846ac83edebdf8cd14ef1/users/ab5a0a0292f04c0db64f5154de1f262c/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:35.669259 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed121092-151d-407b-9777-a69b4c1796ce admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:35.679663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed121092-151d-407b-9777-a69b4c1796ce admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:35.739584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f1f9b-db79-4748-ab24-7c8daf82ed5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:35.740199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f1f9b-db79-4748-ab24-7c8daf82ed5b admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:35.740466 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f1f9b-db79-4748-ab24-7c8daf82ed5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:35.740733 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f1f9b-db79-4748-ab24-7c8daf82ed5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:35.741370 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f1f9b-db79-4748-ab24-7c8daf82ed5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:35.742993 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f1f9b-db79-4748-ab24-7c8daf82ed5b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:35.754062 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1186/2384] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:13:35.758207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38761b19-79ba-49c0-8649-568d0b5528fd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:35.769046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38761b19-79ba-49c0-8649-568d0b5528fd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:35.774482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e08ee2-428d-492f-9dea-84c82d4ef227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:35.775151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e08ee2-428d-492f-9dea-84c82d4ef227 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:35.775421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e08ee2-428d-492f-9dea-84c82d4ef227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:35.775687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e08ee2-428d-492f-9dea-84c82d4ef227 admin admin] PATH_INFO: `/v3/projects/050433d7c3d846ac83edebdf8cd14ef1/users/ab5a0a0292f04c0db64f5154de1f262c/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:35.777759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e08ee2-428d-492f-9dea-84c82d4ef227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=050433d7c3d846ac83edebdf8cd14ef1, user_id=ab5a0a0292f04c0db64f5154de1f262c, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:35.780401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e08ee2-428d-492f-9dea-84c82d4ef227 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:35.795478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1199/2385] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:35 2022] PUT /identity/v3/projects/050433d7c3d846ac83edebdf8cd14ef1/users/ab5a0a0292f04c0db64f5154de1f262c/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:35.806738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49875528-ebde-4c4d-a06b-a1c6f2a083bc None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:35.807025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49875528-ebde-4c4d-a06b-a1c6f2a083bc None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:35.807295 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49875528-ebde-4c4d-a06b-a1c6f2a083bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:35.853121 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-49875528-ebde-4c4d-a06b-a1c6f2a083bc None None] MFA Rules not processed for user `ab5a0a0292f04c0db64f5154de1f262c`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:35.854360 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-49875528-ebde-4c4d-a06b-a1c6f2a083bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:35.864067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:35.931207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1187/2386] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:35.943101 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e08ee2-428d-492f-9dea-84c82d4ef227 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:35.944823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-65e08ee2-428d-492f-9dea-84c82d4ef227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:36.005104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e08ee2-428d-492f-9dea-84c82d4ef227 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:36.008750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7cf2c-58eb-4300-b8e1-762a122a6411 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:36.009351 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7cf2c-58eb-4300-b8e1-762a122a6411 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:36.009625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7cf2c-58eb-4300-b8e1-762a122a6411 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:36.009893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7cf2c-58eb-4300-b8e1-762a122a6411 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:36.019633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7cf2c-58eb-4300-b8e1-762a122a6411 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:36.021038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7cf2c-58eb-4300-b8e1-762a122a6411 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:36.031313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1200/2387] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:36.108426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49875528-ebde-4c4d-a06b-a1c6f2a083bc None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:36.118891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49875528-ebde-4c4d-a06b-a1c6f2a083bc None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:36.122887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d311830-b9ee-48cb-b97b-1de6abbd65bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:36.123491 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d311830-b9ee-48cb-b97b-1de6abbd65bd service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:36.123769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d311830-b9ee-48cb-b97b-1de6abbd65bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:36.124043 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d311830-b9ee-48cb-b97b-1de6abbd65bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:36.185113 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d311830-b9ee-48cb-b97b-1de6abbd65bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:36.186520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d311830-b9ee-48cb-b97b-1de6abbd65bd service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:36.206343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1188/2388] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:38.960411 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7cf2c-58eb-4300-b8e1-762a122a6411 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.002650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:39.025081 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7cf2c-58eb-4300-b8e1-762a122a6411 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.029746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892151ed-c215-409c-9c42-89f9a6dfe18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.030412 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892151ed-c215-409c-9c42-89f9a6dfe18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.030688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892151ed-c215-409c-9c42-89f9a6dfe18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.030986 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892151ed-c215-409c-9c42-89f9a6dfe18d admin admin] PATH_INFO: `/v3/users/61fd0dabc6e44ac4b0ad31619475d6b2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.032724 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892151ed-c215-409c-9c42-89f9a6dfe18d admin admin] RBAC: Authorizing `identity:delete_user(user_id=61fd0dabc6e44ac4b0ad31619475d6b2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.034837 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892151ed-c215-409c-9c42-89f9a6dfe18d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.082636 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-892151ed-c215-409c-9c42-89f9a6dfe18d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61fd0dabc6e44ac4b0ad31619475d6b2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.083294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-892151ed-c215-409c-9c42-89f9a6dfe18d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61fd0dabc6e44ac4b0ad31619475d6b2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.090431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-892151ed-c215-409c-9c42-89f9a6dfe18d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61fd0dabc6e44ac4b0ad31619475d6b2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.099135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-892151ed-c215-409c-9c42-89f9a6dfe18d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61fd0dabc6e44ac4b0ad31619475d6b2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.111873 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-892151ed-c215-409c-9c42-89f9a6dfe18d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61fd0dabc6e44ac4b0ad31619475d6b2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.118934 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1201/2389] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:38 2022] DELETE /identity/v3/users/61fd0dabc6e44ac4b0ad31619475d6b2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:39.123819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170de0e6-50bb-463e-95e3-7d1744b184c7 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.124320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170de0e6-50bb-463e-95e3-7d1744b184c7 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.124420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170de0e6-50bb-463e-95e3-7d1744b184c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.145495 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-170de0e6-50bb-463e-95e3-7d1744b184c7 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:39.146941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-170de0e6-50bb-463e-95e3-7d1744b184c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:39.221164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1189/2390] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:39.231260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892151ed-c215-409c-9c42-89f9a6dfe18d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.232738 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-892151ed-c215-409c-9c42-89f9a6dfe18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:39.297673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892151ed-c215-409c-9c42-89f9a6dfe18d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.302156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472656c7-4a50-47ac-a682-8a3f633a8d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.302767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472656c7-4a50-47ac-a682-8a3f633a8d81 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.303049 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472656c7-4a50-47ac-a682-8a3f633a8d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.303318 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472656c7-4a50-47ac-a682-8a3f633a8d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.312978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472656c7-4a50-47ac-a682-8a3f633a8d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.314519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472656c7-4a50-47ac-a682-8a3f633a8d81 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.324666 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1202/2391] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:39.787419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170de0e6-50bb-463e-95e3-7d1744b184c7 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.788923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-170de0e6-50bb-463e-95e3-7d1744b184c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:39.807465 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170de0e6-50bb-463e-95e3-7d1744b184c7 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.814603 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.815148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.815421 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.815691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4 admin admin] PATH_INFO: `/v3/projects/6dd9d777bc58457391e321c8b9781248` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.816877 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dd9d777bc58457391e321c8b9781248)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.818832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.859711 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:39.867569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dd9d777bc58457391e321c8b9781248', 'request_id': 'req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.875038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dd9d777bc58457391e321c8b9781248', 'request_id': 'req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.881362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dd9d777bc58457391e321c8b9781248', 'request_id': 'req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.894485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4 admin admin] The token cache is being invalidate because project 6dd9d777bc58457391e321c8b9781248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:39.895048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.895814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.897331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1190/2392] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:39 2022] DELETE /identity/v3/projects/6dd9d777bc58457391e321c8b9781248 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:39.901514 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472656c7-4a50-47ac-a682-8a3f633a8d81 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.911257 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472656c7-4a50-47ac-a682-8a3f633a8d81 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.969435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bbdda1-35cc-4f0b-9507-c6074c4b962c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.970204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbdda1-35cc-4f0b-9507-c6074c4b962c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.970375 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbdda1-35cc-4f0b-9507-c6074c4b962c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.970642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbdda1-35cc-4f0b-9507-c6074c4b962c admin admin] PATH_INFO: `/v3/users/ab5a0a0292f04c0db64f5154de1f262c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.971769 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bbdda1-35cc-4f0b-9507-c6074c4b962c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab5a0a0292f04c0db64f5154de1f262c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.973664 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bbdda1-35cc-4f0b-9507-c6074c4b962c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.975122 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:40.022632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e8bbdda1-35cc-4f0b-9507-c6074c4b962c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab5a0a0292f04c0db64f5154de1f262c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:40.023291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e8bbdda1-35cc-4f0b-9507-c6074c4b962c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab5a0a0292f04c0db64f5154de1f262c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:40.030696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e8bbdda1-35cc-4f0b-9507-c6074c4b962c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab5a0a0292f04c0db64f5154de1f262c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:40.039236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e8bbdda1-35cc-4f0b-9507-c6074c4b962c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab5a0a0292f04c0db64f5154de1f262c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:40.051499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e8bbdda1-35cc-4f0b-9507-c6074c4b962c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab5a0a0292f04c0db64f5154de1f262c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:40.057448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1203/2393] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:39 2022] DELETE /identity/v3/users/ab5a0a0292f04c0db64f5154de1f262c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:40.336298 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:40.337949 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:40.351492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523e8d9-0139-4f42-ac8c-795fa0a0fec4 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.413417 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a8602-1263-4a97-a98b-9ce69bddd5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.413976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a8602-1263-4a97-a98b-9ce69bddd5b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.414390 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a8602-1263-4a97-a98b-9ce69bddd5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.414555 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a8602-1263-4a97-a98b-9ce69bddd5b7 admin admin] PATH_INFO: `/v3/projects/050433d7c3d846ac83edebdf8cd14ef1` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.415676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a8602-1263-4a97-a98b-9ce69bddd5b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=050433d7c3d846ac83edebdf8cd14ef1)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.417623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a8602-1263-4a97-a98b-9ce69bddd5b7 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.468968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bf7a8602-1263-4a97-a98b-9ce69bddd5b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '050433d7c3d846ac83edebdf8cd14ef1', 'request_id': 'req-bf7a8602-1263-4a97-a98b-9ce69bddd5b7'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:40.478493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bf7a8602-1263-4a97-a98b-9ce69bddd5b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '050433d7c3d846ac83edebdf8cd14ef1', 'request_id': 'req-bf7a8602-1263-4a97-a98b-9ce69bddd5b7'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:40.487975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bf7a8602-1263-4a97-a98b-9ce69bddd5b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '050433d7c3d846ac83edebdf8cd14ef1', 'request_id': 'req-bf7a8602-1263-4a97-a98b-9ce69bddd5b7'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:40.500657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bf7a8602-1263-4a97-a98b-9ce69bddd5b7 admin admin] The token cache is being invalidate because project 050433d7c3d846ac83edebdf8cd14ef1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:40.501032 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bf7a8602-1263-4a97-a98b-9ce69bddd5b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:40.501780 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bf7a8602-1263-4a97-a98b-9ce69bddd5b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:40.503349 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1191/2394] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:40 2022] DELETE /identity/v3/projects/050433d7c3d846ac83edebdf8cd14ef1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:40.508268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107deded-461d-4ff0-bc93-9201ec720d07 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.508602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107deded-461d-4ff0-bc93-9201ec720d07 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.508904 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107deded-461d-4ff0-bc93-9201ec720d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.528604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-107deded-461d-4ff0-bc93-9201ec720d07 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:40.529815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-107deded-461d-4ff0-bc93-9201ec720d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:40.596895 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1204/2395] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:40.604947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf3e50-879d-4520-a18e-9d2dbfa4fa41 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.605259 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf3e50-879d-4520-a18e-9d2dbfa4fa41 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.605644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf3e50-879d-4520-a18e-9d2dbfa4fa41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.627146 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f2bf3e50-879d-4520-a18e-9d2dbfa4fa41 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:40.628582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f2bf3e50-879d-4520-a18e-9d2dbfa4fa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:40.639961 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1192/2396] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:40.645753 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107deded-461d-4ff0-bc93-9201ec720d07 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:40.655739 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107deded-461d-4ff0-bc93-9201ec720d07 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.660929 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664e0cd-b585-477f-93fb-5b649b34d038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.661531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664e0cd-b585-477f-93fb-5b649b34d038 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.661803 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664e0cd-b585-477f-93fb-5b649b34d038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.662097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664e0cd-b585-477f-93fb-5b649b34d038 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.662772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664e0cd-b585-477f-93fb-5b649b34d038 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:40.663069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664e0cd-b585-477f-93fb-5b649b34d038 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.664701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664e0cd-b585-477f-93fb-5b649b34d038 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.677137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1205/2397] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:13:40.681810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf3e50-879d-4520-a18e-9d2dbfa4fa41 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:40.691326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf3e50-879d-4520-a18e-9d2dbfa4fa41 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.696175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b0020-045c-4be2-9fde-c2256033ae0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.696774 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b0020-045c-4be2-9fde-c2256033ae0a admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.697043 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b0020-045c-4be2-9fde-c2256033ae0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.697308 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b0020-045c-4be2-9fde-c2256033ae0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.697947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b0020-045c-4be2-9fde-c2256033ae0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.699871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b0020-045c-4be2-9fde-c2256033ae0a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.729155 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1193/2398] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:40 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:40.733500 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664e0cd-b585-477f-93fb-5b649b34d038 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:40.744503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664e0cd-b585-477f-93fb-5b649b34d038 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.800504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18934c-a9fc-4418-a63d-60823b41727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.801104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18934c-a9fc-4418-a63d-60823b41727b admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.801378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18934c-a9fc-4418-a63d-60823b41727b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.801646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18934c-a9fc-4418-a63d-60823b41727b admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.802304 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18934c-a9fc-4418-a63d-60823b41727b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.804207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18934c-a9fc-4418-a63d-60823b41727b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.815222 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:40.829499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1206/2399] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:40 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:40.831243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b0020-045c-4be2-9fde-c2256033ae0a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:40.842183 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b0020-045c-4be2-9fde-c2256033ae0a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.847340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dea6b-1eec-47e2-88e0-b690bd77e6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.847962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dea6b-1eec-47e2-88e0-b690bd77e6a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.848237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dea6b-1eec-47e2-88e0-b690bd77e6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.848503 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dea6b-1eec-47e2-88e0-b690bd77e6a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.849148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dea6b-1eec-47e2-88e0-b690bd77e6a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.850770 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dea6b-1eec-47e2-88e0-b690bd77e6a4 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.860506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1194/2400] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:40.864592 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18934c-a9fc-4418-a63d-60823b41727b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:40.874759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18934c-a9fc-4418-a63d-60823b41727b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.879401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721c277-7a2f-4f7a-8d76-da326a2be649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.879981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721c277-7a2f-4f7a-8d76-da326a2be649 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.880257 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721c277-7a2f-4f7a-8d76-da326a2be649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.880524 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721c277-7a2f-4f7a-8d76-da326a2be649 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.881167 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721c277-7a2f-4f7a-8d76-da326a2be649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.882855 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721c277-7a2f-4f7a-8d76-da326a2be649 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.894820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1207/2401] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:40.899355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dea6b-1eec-47e2-88e0-b690bd77e6a4 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:40.910331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dea6b-1eec-47e2-88e0-b690bd77e6a4 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.915170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974189f-4624-4695-9786-e4354b8052d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.915756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974189f-4624-4695-9786-e4354b8052d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.916036 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974189f-4624-4695-9786-e4354b8052d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.916305 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974189f-4624-4695-9786-e4354b8052d1 admin admin] PATH_INFO: `/v3/projects/a1abc034f5304d8e8d4912934a7ca9a1/users/a50b9abc4e0146afa3df5e9f71ffca0c/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.930786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974189f-4624-4695-9786-e4354b8052d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1abc034f5304d8e8d4912934a7ca9a1, user_id=a50b9abc4e0146afa3df5e9f71ffca0c, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.933473 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974189f-4624-4695-9786-e4354b8052d1 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.935659 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:40.949168 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1195/2402] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:40 2022] PUT /identity/v3/projects/a1abc034f5304d8e8d4912934a7ca9a1/users/a50b9abc4e0146afa3df5e9f71ffca0c/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:40.961476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1b37c-ee08-4dde-8265-1438339257fa None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.961768 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1b37c-ee08-4dde-8265-1438339257fa None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.962043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1b37c-ee08-4dde-8265-1438339257fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.991859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7dd1b37c-ee08-4dde-8265-1438339257fa None None] MFA Rules not processed for user `a50b9abc4e0146afa3df5e9f71ffca0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:40.993077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7dd1b37c-ee08-4dde-8265-1438339257fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:41.056888 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1208/2403] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:41.061437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974189f-4624-4695-9786-e4354b8052d1 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:41.122487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974189f-4624-4695-9786-e4354b8052d1 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:41.127241 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b205bb3-9da2-45d0-a882-4ef74209d272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:41.127836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b205bb3-9da2-45d0-a882-4ef74209d272 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:41.128116 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b205bb3-9da2-45d0-a882-4ef74209d272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:41.128386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b205bb3-9da2-45d0-a882-4ef74209d272 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:41.129015 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b205bb3-9da2-45d0-a882-4ef74209d272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:41.130955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b205bb3-9da2-45d0-a882-4ef74209d272 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:41.159261 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1196/2404] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:41 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:41.163524 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1b37c-ee08-4dde-8265-1438339257fa None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:41.174554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1b37c-ee08-4dde-8265-1438339257fa None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:41.231071 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c10fac-2a9f-476e-b19e-705e87be819c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:41.231688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c10fac-2a9f-476e-b19e-705e87be819c admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:41.231994 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c10fac-2a9f-476e-b19e-705e87be819c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:41.232263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c10fac-2a9f-476e-b19e-705e87be819c admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:41.232902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c10fac-2a9f-476e-b19e-705e87be819c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:41.234844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c10fac-2a9f-476e-b19e-705e87be819c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:41.257469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1209/2405] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:41 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:41.261699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b205bb3-9da2-45d0-a882-4ef74209d272 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:41.273778 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b205bb3-9da2-45d0-a882-4ef74209d272 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:41.278719 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8facd4-05d5-4953-a572-ab59338adcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:41.279343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8facd4-05d5-4953-a572-ab59338adcf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:41.279623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8facd4-05d5-4953-a572-ab59338adcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:41.279897 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8facd4-05d5-4953-a572-ab59338adcf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:41.280575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8facd4-05d5-4953-a572-ab59338adcf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:41.282237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8facd4-05d5-4953-a572-ab59338adcf5 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:41.294793 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1197/2406] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:41.299973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c10fac-2a9f-476e-b19e-705e87be819c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:41.309929 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c10fac-2a9f-476e-b19e-705e87be819c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:41.315102 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec4c55-1702-4b2c-9cba-897cbf3648ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:41.315748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec4c55-1702-4b2c-9cba-897cbf3648ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:41.316031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec4c55-1702-4b2c-9cba-897cbf3648ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:41.316306 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec4c55-1702-4b2c-9cba-897cbf3648ba admin admin] PATH_INFO: `/v3/projects/fe14ddf8e804476681006251638f8583/users/d97a88ff9b3d4b7fa6fcc6e5421971a9/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:41.331488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec4c55-1702-4b2c-9cba-897cbf3648ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe14ddf8e804476681006251638f8583, user_id=d97a88ff9b3d4b7fa6fcc6e5421971a9, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:41.334143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec4c55-1702-4b2c-9cba-897cbf3648ba admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:41.347671 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1210/2407] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:41 2022] PUT /identity/v3/projects/fe14ddf8e804476681006251638f8583/users/d97a88ff9b3d4b7fa6fcc6e5421971a9/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:41.351785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8facd4-05d5-4953-a572-ab59338adcf5 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:41.362340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8facd4-05d5-4953-a572-ab59338adcf5 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:41.420439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e02f8-67f2-417a-bc9b-fbe85909944e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:41.421072 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e02f8-67f2-417a-bc9b-fbe85909944e admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:41.421352 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e02f8-67f2-417a-bc9b-fbe85909944e admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:41.421625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e02f8-67f2-417a-bc9b-fbe85909944e admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:41.422328 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e02f8-67f2-417a-bc9b-fbe85909944e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:41.423916 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e02f8-67f2-417a-bc9b-fbe85909944e admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:41.434091 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1198/2408] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:13:41.439088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec4c55-1702-4b2c-9cba-897cbf3648ba admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:41.449781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec4c55-1702-4b2c-9cba-897cbf3648ba admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:41.455622 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d152cd-0ec4-4795-9616-45649b55dddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:41.456516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d152cd-0ec4-4795-9616-45649b55dddf admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:41.456865 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d152cd-0ec4-4795-9616-45649b55dddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:41.457137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d152cd-0ec4-4795-9616-45649b55dddf admin admin] PATH_INFO: `/v3/projects/fe14ddf8e804476681006251638f8583/users/d97a88ff9b3d4b7fa6fcc6e5421971a9/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:41.459121 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d152cd-0ec4-4795-9616-45649b55dddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe14ddf8e804476681006251638f8583, user_id=d97a88ff9b3d4b7fa6fcc6e5421971a9, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:41.461868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d152cd-0ec4-4795-9616-45649b55dddf admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:41.474282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1211/2409] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:41 2022] PUT /identity/v3/projects/fe14ddf8e804476681006251638f8583/users/d97a88ff9b3d4b7fa6fcc6e5421971a9/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:41.485679 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220d06d-9a92-4afa-bb66-a144a45af695 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:41.486045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220d06d-9a92-4afa-bb66-a144a45af695 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:41.486450 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220d06d-9a92-4afa-bb66-a144a45af695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:41.516207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a220d06d-9a92-4afa-bb66-a144a45af695 None None] MFA Rules not processed for user `d97a88ff9b3d4b7fa6fcc6e5421971a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:41.518090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a220d06d-9a92-4afa-bb66-a144a45af695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:41.569911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:41.592334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1199/2410] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:41.603893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d152cd-0ec4-4795-9616-45649b55dddf admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:41.605637 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-94d152cd-0ec4-4795-9616-45649b55dddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:41.638178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:41.667738 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d152cd-0ec4-4795-9616-45649b55dddf admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:41.671551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6bd3f-b14f-4fac-ab6a-db473cb659da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:41.672156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6bd3f-b14f-4fac-ab6a-db473cb659da service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:41.672430 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6bd3f-b14f-4fac-ab6a-db473cb659da service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:41.673249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6bd3f-b14f-4fac-ab6a-db473cb659da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:41.732469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6bd3f-b14f-4fac-ab6a-db473cb659da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:41.733936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6bd3f-b14f-4fac-ab6a-db473cb659da service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:41.743684 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1212/2411] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:45.993835 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220d06d-9a92-4afa-bb66-a144a45af695 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:45.995580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a220d06d-9a92-4afa-bb66-a144a45af695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:46.008441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220d06d-9a92-4afa-bb66-a144a45af695 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:46.012181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19549c6c-042f-4708-bafd-60bfc88bf6d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:46.012790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19549c6c-042f-4708-bafd-60bfc88bf6d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:46.013065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19549c6c-042f-4708-bafd-60bfc88bf6d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:46.013334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19549c6c-042f-4708-bafd-60bfc88bf6d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:46.025779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19549c6c-042f-4708-bafd-60bfc88bf6d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:46.027262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19549c6c-042f-4708-bafd-60bfc88bf6d1 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:46.038335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1200/2412] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:50.995430 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6bd3f-b14f-4fac-ab6a-db473cb659da service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:51.056866 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6bd3f-b14f-4fac-ab6a-db473cb659da service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:51.061646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0074cbd-0824-4b74-a8d2-93208dee86b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:51.062321 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0074cbd-0824-4b74-a8d2-93208dee86b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:51.062601 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0074cbd-0824-4b74-a8d2-93208dee86b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:51.062917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0074cbd-0824-4b74-a8d2-93208dee86b2 admin admin] PATH_INFO: `/v3/users/a50b9abc4e0146afa3df5e9f71ffca0c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:51.064029 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0074cbd-0824-4b74-a8d2-93208dee86b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a50b9abc4e0146afa3df5e9f71ffca0c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:51.066091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0074cbd-0824-4b74-a8d2-93208dee86b2 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:51.112568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f0074cbd-0824-4b74-a8d2-93208dee86b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a50b9abc4e0146afa3df5e9f71ffca0c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:51.113219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f0074cbd-0824-4b74-a8d2-93208dee86b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a50b9abc4e0146afa3df5e9f71ffca0c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:51.120261 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f0074cbd-0824-4b74-a8d2-93208dee86b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a50b9abc4e0146afa3df5e9f71ffca0c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:51.128324 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f0074cbd-0824-4b74-a8d2-93208dee86b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a50b9abc4e0146afa3df5e9f71ffca0c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:51.140164 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f0074cbd-0824-4b74-a8d2-93208dee86b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a50b9abc4e0146afa3df5e9f71ffca0c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:51.142303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:51.147143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1213/2413] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:50 2022] DELETE /identity/v3/users/a50b9abc4e0146afa3df5e9f71ffca0c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:51.151773 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd4857-9be3-4acf-8e4f-faffb59cd3e7 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:51.152277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd4857-9be3-4acf-8e4f-faffb59cd3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:51.152388 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd4857-9be3-4acf-8e4f-faffb59cd3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:51.171939 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-6efd4857-9be3-4acf-8e4f-faffb59cd3e7 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:51.173759 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6efd4857-9be3-4acf-8e4f-faffb59cd3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:51.234772 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1201/2414] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:51.245355 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0074cbd-0824-4b74-a8d2-93208dee86b2 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:51.246763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f0074cbd-0824-4b74-a8d2-93208dee86b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:51.304575 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0074cbd-0824-4b74-a8d2-93208dee86b2 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:51.308205 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f65fd-05fc-48ea-b6fe-a11d26a747cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:51.308802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f65fd-05fc-48ea-b6fe-a11d26a747cd service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:51.309073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f65fd-05fc-48ea-b6fe-a11d26a747cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:51.309340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f65fd-05fc-48ea-b6fe-a11d26a747cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:51.318382 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f65fd-05fc-48ea-b6fe-a11d26a747cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:51.319775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f65fd-05fc-48ea-b6fe-a11d26a747cd service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:51.329471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1214/2415] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:51.602094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd4857-9be3-4acf-8e4f-faffb59cd3e7 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:51.603610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6efd4857-9be3-4acf-8e4f-faffb59cd3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:51.617395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd4857-9be3-4acf-8e4f-faffb59cd3e7 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:51.622582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e4d51-ad68-4c85-8557-dcb22b860513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:51.623210 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e4d51-ad68-4c85-8557-dcb22b860513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:51.623514 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e4d51-ad68-4c85-8557-dcb22b860513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:51.623758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e4d51-ad68-4c85-8557-dcb22b860513 admin admin] PATH_INFO: `/v3/projects/a1abc034f5304d8e8d4912934a7ca9a1` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:51.624998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e4d51-ad68-4c85-8557-dcb22b860513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1abc034f5304d8e8d4912934a7ca9a1)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:51.626937 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e4d51-ad68-4c85-8557-dcb22b860513 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:51.677154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-cc9e4d51-ad68-4c85-8557-dcb22b860513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1abc034f5304d8e8d4912934a7ca9a1', 'request_id': 'req-cc9e4d51-ad68-4c85-8557-dcb22b860513'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:51.684571 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-cc9e4d51-ad68-4c85-8557-dcb22b860513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1abc034f5304d8e8d4912934a7ca9a1', 'request_id': 'req-cc9e4d51-ad68-4c85-8557-dcb22b860513'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:51.690292 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-cc9e4d51-ad68-4c85-8557-dcb22b860513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1abc034f5304d8e8d4912934a7ca9a1', 'request_id': 'req-cc9e4d51-ad68-4c85-8557-dcb22b860513'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:51.701124 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-cc9e4d51-ad68-4c85-8557-dcb22b860513 admin admin] The token cache is being invalidate because project a1abc034f5304d8e8d4912934a7ca9a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:51.701413 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-cc9e4d51-ad68-4c85-8557-dcb22b860513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:51.702056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-cc9e4d51-ad68-4c85-8557-dcb22b860513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:51.704081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1202/2416] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:51 2022] DELETE /identity/v3/projects/a1abc034f5304d8e8d4912934a7ca9a1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:51.708169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f65fd-05fc-48ea-b6fe-a11d26a747cd service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:51.719085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f65fd-05fc-48ea-b6fe-a11d26a747cd service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:51.775318 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b74708-5b88-4e3d-888d-c4c466fe4a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:51.775960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b74708-5b88-4e3d-888d-c4c466fe4a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:51.776239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b74708-5b88-4e3d-888d-c4c466fe4a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:51.776503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b74708-5b88-4e3d-888d-c4c466fe4a7f admin admin] PATH_INFO: `/v3/users/d97a88ff9b3d4b7fa6fcc6e5421971a9` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:51.777617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b74708-5b88-4e3d-888d-c4c466fe4a7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d97a88ff9b3d4b7fa6fcc6e5421971a9)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:51.779537 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b74708-5b88-4e3d-888d-c4c466fe4a7f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:51.839311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-24b74708-5b88-4e3d-888d-c4c466fe4a7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd97a88ff9b3d4b7fa6fcc6e5421971a9'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:51.839995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-24b74708-5b88-4e3d-888d-c4c466fe4a7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd97a88ff9b3d4b7fa6fcc6e5421971a9'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:51.849219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-24b74708-5b88-4e3d-888d-c4c466fe4a7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd97a88ff9b3d4b7fa6fcc6e5421971a9'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:51.858461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-24b74708-5b88-4e3d-888d-c4c466fe4a7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd97a88ff9b3d4b7fa6fcc6e5421971a9'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:51.871433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-24b74708-5b88-4e3d-888d-c4c466fe4a7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd97a88ff9b3d4b7fa6fcc6e5421971a9'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:51.877849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1215/2417] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:51 2022] DELETE /identity/v3/users/d97a88ff9b3d4b7fa6fcc6e5421971a9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:52.148587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e4d51-ad68-4c85-8557-dcb22b860513 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:52.150140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-cc9e4d51-ad68-4c85-8557-dcb22b860513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:52.163583 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e4d51-ad68-4c85-8557-dcb22b860513 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:52.221881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add3672-ecb4-428d-bd6e-976d081be6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:52.222554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add3672-ecb4-428d-bd6e-976d081be6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:52.222833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add3672-ecb4-428d-bd6e-976d081be6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:52.223102 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add3672-ecb4-428d-bd6e-976d081be6d3 admin admin] PATH_INFO: `/v3/projects/fe14ddf8e804476681006251638f8583` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:52.224272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add3672-ecb4-428d-bd6e-976d081be6d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe14ddf8e804476681006251638f8583)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:52.226237 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add3672-ecb4-428d-bd6e-976d081be6d3 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:52.227779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:52.278097 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4add3672-ecb4-428d-bd6e-976d081be6d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe14ddf8e804476681006251638f8583', 'request_id': 'req-4add3672-ecb4-428d-bd6e-976d081be6d3'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:52.285401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4add3672-ecb4-428d-bd6e-976d081be6d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe14ddf8e804476681006251638f8583', 'request_id': 'req-4add3672-ecb4-428d-bd6e-976d081be6d3'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:52.291217 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4add3672-ecb4-428d-bd6e-976d081be6d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe14ddf8e804476681006251638f8583', 'request_id': 'req-4add3672-ecb4-428d-bd6e-976d081be6d3'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:52.303277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4add3672-ecb4-428d-bd6e-976d081be6d3 admin admin] The token cache is being invalidate because project fe14ddf8e804476681006251638f8583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:52.303445 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4add3672-ecb4-428d-bd6e-976d081be6d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:52.304053 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4add3672-ecb4-428d-bd6e-976d081be6d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:52.305553 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1203/2418] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:52 2022] DELETE /identity/v3/projects/fe14ddf8e804476681006251638f8583 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:52.311737 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893fd51-8fd2-4349-92c4-1848e4ba22d6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:52.312251 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893fd51-8fd2-4349-92c4-1848e4ba22d6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:52.312251 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893fd51-8fd2-4349-92c4-1848e4ba22d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:52.332354 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4893fd51-8fd2-4349-92c4-1848e4ba22d6 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:52.333531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4893fd51-8fd2-4349-92c4-1848e4ba22d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:52.349762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:52.393927 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1216/2419] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:52.402462 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a2359-48f3-40e8-bf9a-4dae85799730 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:52.402751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a2359-48f3-40e8-bf9a-4dae85799730 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:52.403055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a2359-48f3-40e8-bf9a-4dae85799730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:52.423990 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-715a2359-48f3-40e8-bf9a-4dae85799730 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:52.425205 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-715a2359-48f3-40e8-bf9a-4dae85799730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:52.433127 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1204/2420] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:52.437254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893fd51-8fd2-4349-92c4-1848e4ba22d6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:52.447503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893fd51-8fd2-4349-92c4-1848e4ba22d6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:52.452043 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea609ab1-c6c9-45f7-84cf-0169845c5889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:52.452635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea609ab1-c6c9-45f7-84cf-0169845c5889 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:52.452910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea609ab1-c6c9-45f7-84cf-0169845c5889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:52.453179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea609ab1-c6c9-45f7-84cf-0169845c5889 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:52.453821 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea609ab1-c6c9-45f7-84cf-0169845c5889 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:52.454139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea609ab1-c6c9-45f7-84cf-0169845c5889 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:52.455728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea609ab1-c6c9-45f7-84cf-0169845c5889 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:52.467090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1217/2421] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:13:52.471987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a2359-48f3-40e8-bf9a-4dae85799730 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:52.481153 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a2359-48f3-40e8-bf9a-4dae85799730 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:52.486304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a604f-777a-427f-b259-3eb9287179dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:52.486895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a604f-777a-427f-b259-3eb9287179dc admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:52.487169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a604f-777a-427f-b259-3eb9287179dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:52.487437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a604f-777a-427f-b259-3eb9287179dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:52.488060 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a604f-777a-427f-b259-3eb9287179dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:52.489981 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a604f-777a-427f-b259-3eb9287179dc admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:52.517185 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1205/2422] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:52 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:52.521433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea609ab1-c6c9-45f7-84cf-0169845c5889 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:52.531333 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea609ab1-c6c9-45f7-84cf-0169845c5889 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:52.586855 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764e6c9-837f-4fb8-a196-eaa14d5886d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:52.587454 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764e6c9-837f-4fb8-a196-eaa14d5886d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:52.587730 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764e6c9-837f-4fb8-a196-eaa14d5886d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:52.587998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764e6c9-837f-4fb8-a196-eaa14d5886d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:52.588617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764e6c9-837f-4fb8-a196-eaa14d5886d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:52.590598 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764e6c9-837f-4fb8-a196-eaa14d5886d0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:52.612441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1218/2423] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:52 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:52.616593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a604f-777a-427f-b259-3eb9287179dc admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:52.627132 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a604f-777a-427f-b259-3eb9287179dc admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:52.631664 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15754d7f-54f5-4d36-b44d-198c57e53902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:52.632256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15754d7f-54f5-4d36-b44d-198c57e53902 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:52.632530 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15754d7f-54f5-4d36-b44d-198c57e53902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:52.632797 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15754d7f-54f5-4d36-b44d-198c57e53902 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:52.633432 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15754d7f-54f5-4d36-b44d-198c57e53902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:52.635066 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15754d7f-54f5-4d36-b44d-198c57e53902 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:52.645404 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1206/2424] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:52.649441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764e6c9-837f-4fb8-a196-eaa14d5886d0 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:52.660141 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764e6c9-837f-4fb8-a196-eaa14d5886d0 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:52.664921 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc53c6-57b6-4362-91f1-08b7cec7e183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:52.665549 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc53c6-57b6-4362-91f1-08b7cec7e183 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:52.665854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc53c6-57b6-4362-91f1-08b7cec7e183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:52.666211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc53c6-57b6-4362-91f1-08b7cec7e183 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:52.666793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc53c6-57b6-4362-91f1-08b7cec7e183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:52.668365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc53c6-57b6-4362-91f1-08b7cec7e183 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:52.679611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1219/2425] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:52.683702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15754d7f-54f5-4d36-b44d-198c57e53902 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:52.693655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15754d7f-54f5-4d36-b44d-198c57e53902 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:52.698257 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3254dc8-21cd-47ba-b51a-a7934884a044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:52.698896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3254dc8-21cd-47ba-b51a-a7934884a044 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:52.699175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3254dc8-21cd-47ba-b51a-a7934884a044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:52.699443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3254dc8-21cd-47ba-b51a-a7934884a044 admin admin] PATH_INFO: `/v3/projects/7ea7086dde23493ca9fe6da014ededa7/users/7f2c02fab325474daa6ec5ca633b2351/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:52.714981 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3254dc8-21cd-47ba-b51a-a7934884a044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea7086dde23493ca9fe6da014ededa7, user_id=7f2c02fab325474daa6ec5ca633b2351, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:52.717649 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3254dc8-21cd-47ba-b51a-a7934884a044 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:52.731990 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1207/2426] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:52 2022] PUT /identity/v3/projects/7ea7086dde23493ca9fe6da014ededa7/users/7f2c02fab325474daa6ec5ca633b2351/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:52.743830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af358c9e-009d-462c-8be4-5a75980b47df None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:52.744362 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af358c9e-009d-462c-8be4-5a75980b47df None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:52.744630 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af358c9e-009d-462c-8be4-5a75980b47df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:52.771408 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-af358c9e-009d-462c-8be4-5a75980b47df None None] MFA Rules not processed for user `7f2c02fab325474daa6ec5ca633b2351`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:52.772650 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-af358c9e-009d-462c-8be4-5a75980b47df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:52.836535 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1220/2427] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:52.846729 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3254dc8-21cd-47ba-b51a-a7934884a044 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:52.908847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3254dc8-21cd-47ba-b51a-a7934884a044 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:52.913523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dac1a-0bc5-4794-b1c5-4bc04b4614ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:52.914147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dac1a-0bc5-4794-b1c5-4bc04b4614ab admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:52.914422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dac1a-0bc5-4794-b1c5-4bc04b4614ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:52.914691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dac1a-0bc5-4794-b1c5-4bc04b4614ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:52.915328 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03dac1a-0bc5-4794-b1c5-4bc04b4614ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:52.917208 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03dac1a-0bc5-4794-b1c5-4bc04b4614ab admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:52.943115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1208/2428] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:52 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:52.947463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af358c9e-009d-462c-8be4-5a75980b47df None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:52.958448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af358c9e-009d-462c-8be4-5a75980b47df None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:52.971466 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:53.019999 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b947c5-7e7c-47e5-a718-2f44cb57b00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:53.020615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b947c5-7e7c-47e5-a718-2f44cb57b00d admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.020893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b947c5-7e7c-47e5-a718-2f44cb57b00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.021166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b947c5-7e7c-47e5-a718-2f44cb57b00d admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.021816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b947c5-7e7c-47e5-a718-2f44cb57b00d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:53.023754 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b947c5-7e7c-47e5-a718-2f44cb57b00d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:53.046347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1221/2429] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:52 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:53.050440 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dac1a-0bc5-4794-b1c5-4bc04b4614ab admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:53.060875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dac1a-0bc5-4794-b1c5-4bc04b4614ab admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:53.065570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbfddc-67ff-4cfc-9489-f16064642184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:53.066194 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbfddc-67ff-4cfc-9489-f16064642184 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.066476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbfddc-67ff-4cfc-9489-f16064642184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.066745 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbfddc-67ff-4cfc-9489-f16064642184 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.067418 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbfddc-67ff-4cfc-9489-f16064642184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:53.068999 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbfddc-67ff-4cfc-9489-f16064642184 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:53.074228 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:53.083610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1209/2430] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:53.084862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b947c5-7e7c-47e5-a718-2f44cb57b00d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:53.095719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b947c5-7e7c-47e5-a718-2f44cb57b00d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:53.100415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48161199-7b4d-4c4e-881a-48747fd08f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:53.101018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48161199-7b4d-4c4e-881a-48747fd08f76 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.101390 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48161199-7b4d-4c4e-881a-48747fd08f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.101550 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48161199-7b4d-4c4e-881a-48747fd08f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.102225 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48161199-7b4d-4c4e-881a-48747fd08f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:53.104018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48161199-7b4d-4c4e-881a-48747fd08f76 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:53.114348 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1222/2431] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:53.119241 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbfddc-67ff-4cfc-9489-f16064642184 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:53.129275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbfddc-67ff-4cfc-9489-f16064642184 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:53.134203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855cbf9-31f0-4a10-881d-fff0cc37fae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:53.134844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855cbf9-31f0-4a10-881d-fff0cc37fae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.135121 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855cbf9-31f0-4a10-881d-fff0cc37fae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.135386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855cbf9-31f0-4a10-881d-fff0cc37fae3 admin admin] PATH_INFO: `/v3/projects/bcc1ab00075e40ba8557bb0665cefa89/users/f264de66fef14c888f28102ff88dce15/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.150591 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855cbf9-31f0-4a10-881d-fff0cc37fae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcc1ab00075e40ba8557bb0665cefa89, user_id=f264de66fef14c888f28102ff88dce15, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:53.153267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855cbf9-31f0-4a10-881d-fff0cc37fae3 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:53.167190 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1210/2432] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:53 2022] PUT /identity/v3/projects/bcc1ab00075e40ba8557bb0665cefa89/users/f264de66fef14c888f28102ff88dce15/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:53.234540 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6302ec9-06c7-4409-8b60-38904bca7ddd None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.234899 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6302ec9-06c7-4409-8b60-38904bca7ddd None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.235207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6302ec9-06c7-4409-8b60-38904bca7ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.263270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d6302ec9-06c7-4409-8b60-38904bca7ddd None None] MFA Rules not processed for user `f264de66fef14c888f28102ff88dce15`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:53.264485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d6302ec9-06c7-4409-8b60-38904bca7ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:53.327875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1223/2433] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:53.333045 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855cbf9-31f0-4a10-881d-fff0cc37fae3 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:53.396372 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855cbf9-31f0-4a10-881d-fff0cc37fae3 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:53.401022 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cd85d-7ffd-478a-bcc1-649d8724b8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:53.401624 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cd85d-7ffd-478a-bcc1-649d8724b8ba admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.401904 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cd85d-7ffd-478a-bcc1-649d8724b8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.402204 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cd85d-7ffd-478a-bcc1-649d8724b8ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.402849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cd85d-7ffd-478a-bcc1-649d8724b8ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:53.404754 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cd85d-7ffd-478a-bcc1-649d8724b8ba admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:53.431954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1211/2434] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:53 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:53.436076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6302ec9-06c7-4409-8b60-38904bca7ddd None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:53.446829 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6302ec9-06c7-4409-8b60-38904bca7ddd None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:53.503016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea72997-4052-4eb1-9bc8-b9999b56bce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:53.503712 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea72997-4052-4eb1-9bc8-b9999b56bce1 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.503929 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea72997-4052-4eb1-9bc8-b9999b56bce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.504204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea72997-4052-4eb1-9bc8-b9999b56bce1 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.504832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea72997-4052-4eb1-9bc8-b9999b56bce1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:53.506798 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea72997-4052-4eb1-9bc8-b9999b56bce1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:53.527949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1224/2435] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:53 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:53.532433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cd85d-7ffd-478a-bcc1-649d8724b8ba admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:53.543052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cd85d-7ffd-478a-bcc1-649d8724b8ba admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:53.548464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60864e6f-43bf-4cc7-b4a9-b502206e0f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:53.549075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60864e6f-43bf-4cc7-b4a9-b502206e0f09 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.549355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60864e6f-43bf-4cc7-b4a9-b502206e0f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.549623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60864e6f-43bf-4cc7-b4a9-b502206e0f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.550347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60864e6f-43bf-4cc7-b4a9-b502206e0f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:53.552197 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60864e6f-43bf-4cc7-b4a9-b502206e0f09 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:53.564197 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1212/2436] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:53.568460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea72997-4052-4eb1-9bc8-b9999b56bce1 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:53.578897 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea72997-4052-4eb1-9bc8-b9999b56bce1 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:53.583499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3eca21-bac6-480b-b253-8f9dcf0b2bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:53.584147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3eca21-bac6-480b-b253-8f9dcf0b2bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.584421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3eca21-bac6-480b-b253-8f9dcf0b2bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.584699 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3eca21-bac6-480b-b253-8f9dcf0b2bcd admin admin] PATH_INFO: `/v3/projects/44a67cb769b2411a9268e724add90a0c/users/d609e8c68315488b903080b86f9f7a5b/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.598416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:53.602009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3eca21-bac6-480b-b253-8f9dcf0b2bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a67cb769b2411a9268e724add90a0c, user_id=d609e8c68315488b903080b86f9f7a5b, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:53.605191 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3eca21-bac6-480b-b253-8f9dcf0b2bcd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:53.619293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1225/2437] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:53 2022] PUT /identity/v3/projects/44a67cb769b2411a9268e724add90a0c/users/d609e8c68315488b903080b86f9f7a5b/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:53.623410 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60864e6f-43bf-4cc7-b4a9-b502206e0f09 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:53.633322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60864e6f-43bf-4cc7-b4a9-b502206e0f09 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:53.690336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee019cdc-bad7-4298-a73f-b098b50ce0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:53.690950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee019cdc-bad7-4298-a73f-b098b50ce0ff admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.691226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee019cdc-bad7-4298-a73f-b098b50ce0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.691495 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee019cdc-bad7-4298-a73f-b098b50ce0ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.692140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee019cdc-bad7-4298-a73f-b098b50ce0ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:53.693752 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee019cdc-bad7-4298-a73f-b098b50ce0ff admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:53.704018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1213/2438] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:53.708309 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3eca21-bac6-480b-b253-8f9dcf0b2bcd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:53.718842 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3eca21-bac6-480b-b253-8f9dcf0b2bcd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:53.724359 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34bed3-810c-4f02-a591-8c88828b587f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:53.725103 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34bed3-810c-4f02-a591-8c88828b587f admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.725479 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34bed3-810c-4f02-a591-8c88828b587f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.725843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34bed3-810c-4f02-a591-8c88828b587f admin admin] PATH_INFO: `/v3/projects/44a67cb769b2411a9268e724add90a0c/users/d609e8c68315488b903080b86f9f7a5b/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.728017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34bed3-810c-4f02-a591-8c88828b587f admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a67cb769b2411a9268e724add90a0c, user_id=d609e8c68315488b903080b86f9f7a5b, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:53.730765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34bed3-810c-4f02-a591-8c88828b587f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:53.740821 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1226/2439] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:53 2022] PUT /identity/v3/projects/44a67cb769b2411a9268e724add90a0c/users/d609e8c68315488b903080b86f9f7a5b/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:53.751591 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f10287-0967-4390-af98-73956c2c449e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.751959 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f10287-0967-4390-af98-73956c2c449e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.752328 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f10287-0967-4390-af98-73956c2c449e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.779977 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-39f10287-0967-4390-af98-73956c2c449e None None] MFA Rules not processed for user `d609e8c68315488b903080b86f9f7a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:53.781176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-39f10287-0967-4390-af98-73956c2c449e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:53.814130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:53.850974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1214/2440] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:13:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:53.862971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34bed3-810c-4f02-a591-8c88828b587f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:53.864763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cf34bed3-810c-4f02-a591-8c88828b587f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:53.922232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34bed3-810c-4f02-a591-8c88828b587f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:53.925711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379cd49d-7248-40d2-8ad1-dc682d482d3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:53.926339 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cd49d-7248-40d2-8ad1-dc682d482d3d service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.926615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cd49d-7248-40d2-8ad1-dc682d482d3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.926883 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cd49d-7248-40d2-8ad1-dc682d482d3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.982182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379cd49d-7248-40d2-8ad1-dc682d482d3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:53.983589 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379cd49d-7248-40d2-8ad1-dc682d482d3d service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:53.992404 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1227/2441] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:55.515271 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f10287-0967-4390-af98-73956c2c449e None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:55.517351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-39f10287-0967-4390-af98-73956c2c449e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:55.529095 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f10287-0967-4390-af98-73956c2c449e None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:55.532682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053185a4-88d7-4a8e-a1dc-05e9bcb47ddc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:55.533278 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053185a4-88d7-4a8e-a1dc-05e9bcb47ddc service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:55.533548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053185a4-88d7-4a8e-a1dc-05e9bcb47ddc service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:55.533813 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053185a4-88d7-4a8e-a1dc-05e9bcb47ddc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:55.589832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053185a4-88d7-4a8e-a1dc-05e9bcb47ddc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:55.591321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053185a4-88d7-4a8e-a1dc-05e9bcb47ddc service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:55.600424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1215/2442] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:56.151589 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379cd49d-7248-40d2-8ad1-dc682d482d3d service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:56.160133 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379cd49d-7248-40d2-8ad1-dc682d482d3d service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:56.163747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0f0fe-af60-4a31-b6b6-f5d34cd0da1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:56.164343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0f0fe-af60-4a31-b6b6-f5d34cd0da1c service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:56.164616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0f0fe-af60-4a31-b6b6-f5d34cd0da1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:56.164893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0f0fe-af60-4a31-b6b6-f5d34cd0da1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:56.174820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0f0fe-af60-4a31-b6b6-f5d34cd0da1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:56.176227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0f0fe-af60-4a31-b6b6-f5d34cd0da1c service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:56.185838 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1228/2443] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:14:00.065913 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053185a4-88d7-4a8e-a1dc-05e9bcb47ddc service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:00.129056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053185a4-88d7-4a8e-a1dc-05e9bcb47ddc service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:00.133812 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5717c7-1a3f-47c2-abfa-3f67b7eb3fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:00.134470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5717c7-1a3f-47c2-abfa-3f67b7eb3fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:00.134753 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5717c7-1a3f-47c2-abfa-3f67b7eb3fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:00.135023 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5717c7-1a3f-47c2-abfa-3f67b7eb3fce admin admin] PATH_INFO: `/v3/users/7f2c02fab325474daa6ec5ca633b2351` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:00.136145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5717c7-1a3f-47c2-abfa-3f67b7eb3fce admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2c02fab325474daa6ec5ca633b2351)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:00.138088 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5717c7-1a3f-47c2-abfa-3f67b7eb3fce admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:00.184890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4c5717c7-1a3f-47c2-abfa-3f67b7eb3fce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2c02fab325474daa6ec5ca633b2351'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:00.192493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4c5717c7-1a3f-47c2-abfa-3f67b7eb3fce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2c02fab325474daa6ec5ca633b2351'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:00.209018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4c5717c7-1a3f-47c2-abfa-3f67b7eb3fce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2c02fab325474daa6ec5ca633b2351'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:00.210472 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:00.218479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4c5717c7-1a3f-47c2-abfa-3f67b7eb3fce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2c02fab325474daa6ec5ca633b2351'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:00.219998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4c5717c7-1a3f-47c2-abfa-3f67b7eb3fce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2c02fab325474daa6ec5ca633b2351'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:00.227050 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1216/2444] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:00 2022] DELETE /identity/v3/users/7f2c02fab325474daa6ec5ca633b2351 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:00.231573 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33057ae-dbb1-4ac2-a4ea-616fccbe514b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:00.232020 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33057ae-dbb1-4ac2-a4ea-616fccbe514b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:00.232107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33057ae-dbb1-4ac2-a4ea-616fccbe514b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:00.251629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f33057ae-dbb1-4ac2-a4ea-616fccbe514b None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:00.252756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f33057ae-dbb1-4ac2-a4ea-616fccbe514b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:00.309812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1229/2445] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:00.319501 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5717c7-1a3f-47c2-abfa-3f67b7eb3fce admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:00.320967 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-4c5717c7-1a3f-47c2-abfa-3f67b7eb3fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:00.382140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5717c7-1a3f-47c2-abfa-3f67b7eb3fce admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:00.385837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790680e2-f72f-4fa0-bb08-87eaebb740bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:00.386476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790680e2-f72f-4fa0-bb08-87eaebb740bf service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:00.386756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790680e2-f72f-4fa0-bb08-87eaebb740bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:00.387027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790680e2-f72f-4fa0-bb08-87eaebb740bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:00.397741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790680e2-f72f-4fa0-bb08-87eaebb740bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:00.399217 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790680e2-f72f-4fa0-bb08-87eaebb740bf service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:00.409647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1217/2446] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:00.687053 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33057ae-dbb1-4ac2-a4ea-616fccbe514b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:00.688662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f33057ae-dbb1-4ac2-a4ea-616fccbe514b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:00.701819 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33057ae-dbb1-4ac2-a4ea-616fccbe514b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:00.707424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1420a-908a-4b38-b5eb-3a580d2091f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:00.708058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1420a-908a-4b38-b5eb-3a580d2091f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:00.708329 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1420a-908a-4b38-b5eb-3a580d2091f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:00.708594 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1420a-908a-4b38-b5eb-3a580d2091f4 admin admin] PATH_INFO: `/v3/projects/7ea7086dde23493ca9fe6da014ededa7` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:00.709735 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1420a-908a-4b38-b5eb-3a580d2091f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ea7086dde23493ca9fe6da014ededa7)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:00.711859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1420a-908a-4b38-b5eb-3a580d2091f4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:00.729247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:00.760973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-66a1420a-908a-4b38-b5eb-3a580d2091f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ea7086dde23493ca9fe6da014ededa7', 'request_id': 'req-66a1420a-908a-4b38-b5eb-3a580d2091f4'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:00.768963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-66a1420a-908a-4b38-b5eb-3a580d2091f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ea7086dde23493ca9fe6da014ededa7', 'request_id': 'req-66a1420a-908a-4b38-b5eb-3a580d2091f4'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:00.775285 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-66a1420a-908a-4b38-b5eb-3a580d2091f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ea7086dde23493ca9fe6da014ededa7', 'request_id': 'req-66a1420a-908a-4b38-b5eb-3a580d2091f4'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:00.786713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-66a1420a-908a-4b38-b5eb-3a580d2091f4 admin admin] The token cache is being invalidate because project 7ea7086dde23493ca9fe6da014ededa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:00.786999 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-66a1420a-908a-4b38-b5eb-3a580d2091f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:00.787644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-66a1420a-908a-4b38-b5eb-3a580d2091f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:00.789146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1230/2447] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:00 2022] DELETE /identity/v3/projects/7ea7086dde23493ca9fe6da014ededa7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:14:00.793229 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790680e2-f72f-4fa0-bb08-87eaebb740bf service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:00.802772 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790680e2-f72f-4fa0-bb08-87eaebb740bf service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:00.863740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f4225-34e2-457b-a376-d16e7299aab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:00.864375 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f4225-34e2-457b-a376-d16e7299aab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:00.864651 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f4225-34e2-457b-a376-d16e7299aab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:00.864920 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f4225-34e2-457b-a376-d16e7299aab0 admin admin] PATH_INFO: `/v3/users/f264de66fef14c888f28102ff88dce15` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:00.866035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f4225-34e2-457b-a376-d16e7299aab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f264de66fef14c888f28102ff88dce15)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:00.867976 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f4225-34e2-457b-a376-d16e7299aab0 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:00.915177 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5f4f4225-34e2-457b-a376-d16e7299aab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f264de66fef14c888f28102ff88dce15'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:00.922495 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5f4f4225-34e2-457b-a376-d16e7299aab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f264de66fef14c888f28102ff88dce15'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:00.934075 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5f4f4225-34e2-457b-a376-d16e7299aab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f264de66fef14c888f28102ff88dce15'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:00.941313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5f4f4225-34e2-457b-a376-d16e7299aab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f264de66fef14c888f28102ff88dce15'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:00.941952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5f4f4225-34e2-457b-a376-d16e7299aab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f264de66fef14c888f28102ff88dce15'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:00.947667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1218/2448] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:00 2022] DELETE /identity/v3/users/f264de66fef14c888f28102ff88dce15 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:01.231413 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1420a-908a-4b38-b5eb-3a580d2091f4 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:01.232905 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-66a1420a-908a-4b38-b5eb-3a580d2091f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:01.246979 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1420a-908a-4b38-b5eb-3a580d2091f4 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:01.304728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0533b9-8d92-4dc7-9799-43ea788afdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:01.305285 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0533b9-8d92-4dc7-9799-43ea788afdc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:01.305564 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0533b9-8d92-4dc7-9799-43ea788afdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:01.305833 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0533b9-8d92-4dc7-9799-43ea788afdc2 admin admin] PATH_INFO: `/v3/projects/bcc1ab00075e40ba8557bb0665cefa89` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:01.306984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0533b9-8d92-4dc7-9799-43ea788afdc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcc1ab00075e40ba8557bb0665cefa89)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:01.308878 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0533b9-8d92-4dc7-9799-43ea788afdc2 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:01.354859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3e0533b9-8d92-4dc7-9799-43ea788afdc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcc1ab00075e40ba8557bb0665cefa89', 'request_id': 'req-3e0533b9-8d92-4dc7-9799-43ea788afdc2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.362366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3e0533b9-8d92-4dc7-9799-43ea788afdc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcc1ab00075e40ba8557bb0665cefa89', 'request_id': 'req-3e0533b9-8d92-4dc7-9799-43ea788afdc2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.368027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3e0533b9-8d92-4dc7-9799-43ea788afdc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcc1ab00075e40ba8557bb0665cefa89', 'request_id': 'req-3e0533b9-8d92-4dc7-9799-43ea788afdc2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.379623 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3e0533b9-8d92-4dc7-9799-43ea788afdc2 admin admin] The token cache is being invalidate because project bcc1ab00075e40ba8557bb0665cefa89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:01.379939 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3e0533b9-8d92-4dc7-9799-43ea788afdc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.380606 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3e0533b9-8d92-4dc7-9799-43ea788afdc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.382175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1231/2449] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:01 2022] DELETE /identity/v3/projects/bcc1ab00075e40ba8557bb0665cefa89 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:01.386239 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f4225-34e2-457b-a376-d16e7299aab0 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:01.387654 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-5f4f4225-34e2-457b-a376-d16e7299aab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:01.400844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f4225-34e2-457b-a376-d16e7299aab0 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:01.408993 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:01.463788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f4802-8165-480d-a2d6-00908bf86b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:01.464496 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f4802-8165-480d-a2d6-00908bf86b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:01.464775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f4802-8165-480d-a2d6-00908bf86b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:01.465042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f4802-8165-480d-a2d6-00908bf86b57 admin admin] PATH_INFO: `/v3/users/d609e8c68315488b903080b86f9f7a5b` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:01.466864 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f4802-8165-480d-a2d6-00908bf86b57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d609e8c68315488b903080b86f9f7a5b)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:01.469650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f4802-8165-480d-a2d6-00908bf86b57 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:01.518413 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-713f4802-8165-480d-a2d6-00908bf86b57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd609e8c68315488b903080b86f9f7a5b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.525430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-713f4802-8165-480d-a2d6-00908bf86b57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd609e8c68315488b903080b86f9f7a5b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.537460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-713f4802-8165-480d-a2d6-00908bf86b57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd609e8c68315488b903080b86f9f7a5b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.544678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-713f4802-8165-480d-a2d6-00908bf86b57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd609e8c68315488b903080b86f9f7a5b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.545348 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-713f4802-8165-480d-a2d6-00908bf86b57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd609e8c68315488b903080b86f9f7a5b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.551288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1219/2450] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:01 2022] DELETE /identity/v3/users/d609e8c68315488b903080b86f9f7a5b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:01.825313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0533b9-8d92-4dc7-9799-43ea788afdc2 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:01.826785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3e0533b9-8d92-4dc7-9799-43ea788afdc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:01.841582 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0533b9-8d92-4dc7-9799-43ea788afdc2 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:01.872804 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:01.906939 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b74e3-4afb-4e4c-a6de-174c6ebe60e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:01.907594 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b74e3-4afb-4e4c-a6de-174c6ebe60e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:01.907868 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b74e3-4afb-4e4c-a6de-174c6ebe60e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:01.908137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b74e3-4afb-4e4c-a6de-174c6ebe60e0 admin admin] PATH_INFO: `/v3/projects/44a67cb769b2411a9268e724add90a0c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:01.909282 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b74e3-4afb-4e4c-a6de-174c6ebe60e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a67cb769b2411a9268e724add90a0c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:01.911260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b74e3-4afb-4e4c-a6de-174c6ebe60e0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:01.959149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3e7b74e3-4afb-4e4c-a6de-174c6ebe60e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a67cb769b2411a9268e724add90a0c', 'request_id': 'req-3e7b74e3-4afb-4e4c-a6de-174c6ebe60e0'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.967098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3e7b74e3-4afb-4e4c-a6de-174c6ebe60e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a67cb769b2411a9268e724add90a0c', 'request_id': 'req-3e7b74e3-4afb-4e4c-a6de-174c6ebe60e0'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.973039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3e7b74e3-4afb-4e4c-a6de-174c6ebe60e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a67cb769b2411a9268e724add90a0c', 'request_id': 'req-3e7b74e3-4afb-4e4c-a6de-174c6ebe60e0'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.984142 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3e7b74e3-4afb-4e4c-a6de-174c6ebe60e0 admin admin] The token cache is being invalidate because project 44a67cb769b2411a9268e724add90a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:01.984974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3e7b74e3-4afb-4e4c-a6de-174c6ebe60e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.985615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3e7b74e3-4afb-4e4c-a6de-174c6ebe60e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.987387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1232/2451] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:01 2022] DELETE /identity/v3/projects/44a67cb769b2411a9268e724add90a0c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:01.992179 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cff8b-72cb-466b-96ff-463b22ccec61 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:01.992463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cff8b-72cb-466b-96ff-463b22ccec61 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:01.992736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cff8b-72cb-466b-96ff-463b22ccec61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:02.013116 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-e27cff8b-72cb-466b-96ff-463b22ccec61 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:02.014354 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e27cff8b-72cb-466b-96ff-463b22ccec61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:02.075952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1220/2452] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:02.085370 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfefcf-8a26-4b0a-b5d0-27b17a6ad847 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:02.085707 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfefcf-8a26-4b0a-b5d0-27b17a6ad847 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:02.086156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfefcf-8a26-4b0a-b5d0-27b17a6ad847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:02.107264 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4fcfefcf-8a26-4b0a-b5d0-27b17a6ad847 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:02.108483 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4fcfefcf-8a26-4b0a-b5d0-27b17a6ad847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:02.117037 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1233/2453] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:02.122058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27cff8b-72cb-466b-96ff-463b22ccec61 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:02.132365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27cff8b-72cb-466b-96ff-463b22ccec61 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:02.137635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11727e-4976-4409-8411-caba2a3dd8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:02.138253 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11727e-4976-4409-8411-caba2a3dd8cb admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:02.138529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11727e-4976-4409-8411-caba2a3dd8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:02.138800 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11727e-4976-4409-8411-caba2a3dd8cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:02.139443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11727e-4976-4409-8411-caba2a3dd8cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:14:02.139737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11727e-4976-4409-8411-caba2a3dd8cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:02.141315 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11727e-4976-4409-8411-caba2a3dd8cb admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:02.152667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1221/2454] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:14:02.157270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfefcf-8a26-4b0a-b5d0-27b17a6ad847 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:02.166565 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfefcf-8a26-4b0a-b5d0-27b17a6ad847 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:02.171270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188ce97-d8be-4a08-80cd-4ebc5a1b6014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:02.173028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188ce97-d8be-4a08-80cd-4ebc5a1b6014 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:02.173316 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188ce97-d8be-4a08-80cd-4ebc5a1b6014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:02.173582 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188ce97-d8be-4a08-80cd-4ebc5a1b6014 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:02.174248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188ce97-d8be-4a08-80cd-4ebc5a1b6014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:02.176147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188ce97-d8be-4a08-80cd-4ebc5a1b6014 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:02.205786 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1234/2455] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:02 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:02.209938 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11727e-4976-4409-8411-caba2a3dd8cb admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:02.220516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11727e-4976-4409-8411-caba2a3dd8cb admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:02.233616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:02.278865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7ba95-bd66-42c1-a192-e7b5c8d38e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:02.279465 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7ba95-bd66-42c1-a192-e7b5c8d38e65 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:02.279744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7ba95-bd66-42c1-a192-e7b5c8d38e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:02.280013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7ba95-bd66-42c1-a192-e7b5c8d38e65 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:02.280630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7ba95-bd66-42c1-a192-e7b5c8d38e65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:02.283094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7ba95-bd66-42c1-a192-e7b5c8d38e65 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:02.304821 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1222/2456] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:02 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:14:02.308964 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188ce97-d8be-4a08-80cd-4ebc5a1b6014 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:02.319383 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188ce97-d8be-4a08-80cd-4ebc5a1b6014 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:02.324155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb54826-f3b9-45e3-8e57-d38cf00ac6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:02.324744 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb54826-f3b9-45e3-8e57-d38cf00ac6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:02.325022 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb54826-f3b9-45e3-8e57-d38cf00ac6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:02.325288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb54826-f3b9-45e3-8e57-d38cf00ac6e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:02.325917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb54826-f3b9-45e3-8e57-d38cf00ac6e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:02.327607 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb54826-f3b9-45e3-8e57-d38cf00ac6e6 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:02.337423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1235/2457] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:02.341373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7ba95-bd66-42c1-a192-e7b5c8d38e65 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:02.351923 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7ba95-bd66-42c1-a192-e7b5c8d38e65 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:02.356607 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a872e18-ddce-442a-9531-0020c9451599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:02.357216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a872e18-ddce-442a-9531-0020c9451599 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:02.357494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a872e18-ddce-442a-9531-0020c9451599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:02.357770 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a872e18-ddce-442a-9531-0020c9451599 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:02.358435 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a872e18-ddce-442a-9531-0020c9451599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:02.360738 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a872e18-ddce-442a-9531-0020c9451599 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:02.371606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1223/2458] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:02.376725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb54826-f3b9-45e3-8e57-d38cf00ac6e6 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:02.386834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb54826-f3b9-45e3-8e57-d38cf00ac6e6 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:02.392285 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f599ad-29ec-48a6-908f-efeb019fcfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:02.392954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f599ad-29ec-48a6-908f-efeb019fcfd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:02.393224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f599ad-29ec-48a6-908f-efeb019fcfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:02.393492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f599ad-29ec-48a6-908f-efeb019fcfd2 admin admin] PATH_INFO: `/v3/projects/91e1cf881ca84378b2ba4c274e534d06/users/748dbf4389964a7ab6bb4247d94c699a/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:02.409305 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f599ad-29ec-48a6-908f-efeb019fcfd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e1cf881ca84378b2ba4c274e534d06, user_id=748dbf4389964a7ab6bb4247d94c699a, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:02.412026 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f599ad-29ec-48a6-908f-efeb019fcfd2 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:02.426092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1236/2459] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:02 2022] PUT /identity/v3/projects/91e1cf881ca84378b2ba4c274e534d06/users/748dbf4389964a7ab6bb4247d94c699a/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:02.436493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc4b48-f678-42e2-9c76-a68d2289c23c None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:02.436994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc4b48-f678-42e2-9c76-a68d2289c23c None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:02.437109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc4b48-f678-42e2-9c76-a68d2289c23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:02.464071 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-e1cc4b48-f678-42e2-9c76-a68d2289c23c None None] MFA Rules not processed for user `748dbf4389964a7ab6bb4247d94c699a`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:02.465282 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e1cc4b48-f678-42e2-9c76-a68d2289c23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:02.530759 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1224/2460] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:14:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:02.542601 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f599ad-29ec-48a6-908f-efeb019fcfd2 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:02.544288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f2f599ad-29ec-48a6-908f-efeb019fcfd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:02.600954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f599ad-29ec-48a6-908f-efeb019fcfd2 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:02.604648 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f57dae-b48a-406a-a058-0e21b343d5e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:02.605248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f57dae-b48a-406a-a058-0e21b343d5e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:02.605524 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f57dae-b48a-406a-a058-0e21b343d5e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:02.605793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f57dae-b48a-406a-a058-0e21b343d5e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:02.613981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f57dae-b48a-406a-a058-0e21b343d5e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:02.615435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f57dae-b48a-406a-a058-0e21b343d5e7 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:02.624290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1237/2461] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:12.527731 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=58,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:12.620343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:29.747052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc4b48-f678-42e2-9c76-a68d2289c23c None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:29.748515 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e1cc4b48-f678-42e2-9c76-a68d2289c23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:29.915928 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc4b48-f678-42e2-9c76-a68d2289c23c None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:29.920880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e59a8-bf08-448a-8af5-90b022514206 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:29.921479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e59a8-bf08-448a-8af5-90b022514206 service placement] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:29.921759 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e59a8-bf08-448a-8af5-90b022514206 service placement] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:29.922016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e59a8-bf08-448a-8af5-90b022514206 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:29.930644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-327e59a8-bf08-448a-8af5-90b022514206 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:29.946968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e59a8-bf08-448a-8af5-90b022514206 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:29.948569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e59a8-bf08-448a-8af5-90b022514206 service placement] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:29.960469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1225/2462] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:32.991666 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6e922-8b7e-4afc-9d74-9bf053646469 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:32.992211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6e922-8b7e-4afc-9d74-9bf053646469 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:32.992211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6e922-8b7e-4afc-9d74-9bf053646469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:33.015264 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d7d6e922-8b7e-4afc-9d74-9bf053646469 None None] MFA Rules not processed for user `16c99b5db9794dff83ea1b67fd238655`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:33.016692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d7d6e922-8b7e-4afc-9d74-9bf053646469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:33.077358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1238/2463] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:14:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:39.956355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=63 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:43.070877 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=25 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:48.855024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e59a8-bf08-448a-8af5-90b022514206 service placement] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:48.943869 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e59a8-bf08-448a-8af5-90b022514206 service placement] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:48.948573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5991e9-2e70-4103-98cf-bf0f2256029a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:48.949230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5991e9-2e70-4103-98cf-bf0f2256029a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:48.949508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5991e9-2e70-4103-98cf-bf0f2256029a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:48.949781 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5991e9-2e70-4103-98cf-bf0f2256029a admin admin] PATH_INFO: `/v3/users/748dbf4389964a7ab6bb4247d94c699a` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:48.951012 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5991e9-2e70-4103-98cf-bf0f2256029a admin admin] RBAC: Authorizing `identity:delete_user(user_id=748dbf4389964a7ab6bb4247d94c699a)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:48.952968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5991e9-2e70-4103-98cf-bf0f2256029a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:49.003718 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bc5991e9-2e70-4103-98cf-bf0f2256029a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '748dbf4389964a7ab6bb4247d94c699a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:49.014053 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bc5991e9-2e70-4103-98cf-bf0f2256029a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '748dbf4389964a7ab6bb4247d94c699a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:49.027498 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bc5991e9-2e70-4103-98cf-bf0f2256029a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '748dbf4389964a7ab6bb4247d94c699a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:49.035633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bc5991e9-2e70-4103-98cf-bf0f2256029a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '748dbf4389964a7ab6bb4247d94c699a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:49.036362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-bc5991e9-2e70-4103-98cf-bf0f2256029a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '748dbf4389964a7ab6bb4247d94c699a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:49.042350 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1226/2464] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:48 2022] DELETE /identity/v3/users/748dbf4389964a7ab6bb4247d94c699a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:49.047545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c46961-f2e2-40da-9830-af966e91942f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:49.048049 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c46961-f2e2-40da-9830-af966e91942f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:49.048136 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c46961-f2e2-40da-9830-af966e91942f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:49.090848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-93c46961-f2e2-40da-9830-af966e91942f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:49.092058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-93c46961-f2e2-40da-9830-af966e91942f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:49.150925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1239/2465] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:49.160645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5991e9-2e70-4103-98cf-bf0f2256029a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:49.162052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-bc5991e9-2e70-4103-98cf-bf0f2256029a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:49.231609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:49.243687 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5991e9-2e70-4103-98cf-bf0f2256029a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:49.247436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f800e36-2dcb-4689-a6ab-46ffd5889db6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:49.248040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f800e36-2dcb-4689-a6ab-46ffd5889db6 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:49.248317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f800e36-2dcb-4689-a6ab-46ffd5889db6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:49.248585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f800e36-2dcb-4689-a6ab-46ffd5889db6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:49.258657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f800e36-2dcb-4689-a6ab-46ffd5889db6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:49.260091 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f800e36-2dcb-4689-a6ab-46ffd5889db6 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:49.269583 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1227/2466] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:49.535489 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c46961-f2e2-40da-9830-af966e91942f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:49.537017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-93c46961-f2e2-40da-9830-af966e91942f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:49.549636 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c46961-f2e2-40da-9830-af966e91942f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:49.554269 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7ff37-b538-4469-aae1-76e51f4c10f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:49.554905 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7ff37-b538-4469-aae1-76e51f4c10f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:49.555181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7ff37-b538-4469-aae1-76e51f4c10f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:49.555455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7ff37-b538-4469-aae1-76e51f4c10f1 admin admin] PATH_INFO: `/v3/projects/91e1cf881ca84378b2ba4c274e534d06` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:49.556610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7ff37-b538-4469-aae1-76e51f4c10f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91e1cf881ca84378b2ba4c274e534d06)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:49.558558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7ff37-b538-4469-aae1-76e51f4c10f1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:49.609418 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-58e7ff37-b538-4469-aae1-76e51f4c10f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e1cf881ca84378b2ba4c274e534d06', 'request_id': 'req-58e7ff37-b538-4469-aae1-76e51f4c10f1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:49.616823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-58e7ff37-b538-4469-aae1-76e51f4c10f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e1cf881ca84378b2ba4c274e534d06', 'request_id': 'req-58e7ff37-b538-4469-aae1-76e51f4c10f1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:49.623059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-58e7ff37-b538-4469-aae1-76e51f4c10f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e1cf881ca84378b2ba4c274e534d06', 'request_id': 'req-58e7ff37-b538-4469-aae1-76e51f4c10f1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:49.634629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-58e7ff37-b538-4469-aae1-76e51f4c10f1 admin admin] The token cache is being invalidate because project 91e1cf881ca84378b2ba4c274e534d06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:49.634919 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-58e7ff37-b538-4469-aae1-76e51f4c10f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:49.635553 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-58e7ff37-b538-4469-aae1-76e51f4c10f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:49.637096 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1240/2467] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:49 2022] DELETE /identity/v3/projects/91e1cf881ca84378b2ba4c274e534d06 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:49.642686 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545924d7-0354-488e-b2ae-1b09272f9639 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:49.642964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545924d7-0354-488e-b2ae-1b09272f9639 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:49.643229 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545924d7-0354-488e-b2ae-1b09272f9639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:49.690922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-545924d7-0354-488e-b2ae-1b09272f9639 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:49.692013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-545924d7-0354-488e-b2ae-1b09272f9639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:49.763296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1228/2468] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:49.773095 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592c89c-b460-4f1a-a48d-e5dfddb20369 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:49.773379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592c89c-b460-4f1a-a48d-e5dfddb20369 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:49.773646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592c89c-b460-4f1a-a48d-e5dfddb20369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:49.794380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1592c89c-b460-4f1a-a48d-e5dfddb20369 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:49.795623 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1592c89c-b460-4f1a-a48d-e5dfddb20369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:49.805753 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1241/2469] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:49.809931 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545924d7-0354-488e-b2ae-1b09272f9639 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:49.820321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545924d7-0354-488e-b2ae-1b09272f9639 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:49.825268 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ea01c-e89a-4c48-b144-ba9c10471c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:49.826027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ea01c-e89a-4c48-b144-ba9c10471c00 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:49.826459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ea01c-e89a-4c48-b144-ba9c10471c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:49.826818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ea01c-e89a-4c48-b144-ba9c10471c00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:49.827488 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ea01c-e89a-4c48-b144-ba9c10471c00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:14:49.827793 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ea01c-e89a-4c48-b144-ba9c10471c00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:49.829536 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ea01c-e89a-4c48-b144-ba9c10471c00 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:49.841903 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1229/2470] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:14:49.846644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592c89c-b460-4f1a-a48d-e5dfddb20369 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:49.856406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592c89c-b460-4f1a-a48d-e5dfddb20369 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:49.862139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059601b3-2c47-49ad-8680-34325123217f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:49.862723 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059601b3-2c47-49ad-8680-34325123217f admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:49.862997 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059601b3-2c47-49ad-8680-34325123217f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:49.863270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059601b3-2c47-49ad-8680-34325123217f admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:49.863898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059601b3-2c47-49ad-8680-34325123217f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:49.865778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059601b3-2c47-49ad-8680-34325123217f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:49.895617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1242/2471] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:49 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:49.900024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ea01c-e89a-4c48-b144-ba9c10471c00 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:49.910146 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ea01c-e89a-4c48-b144-ba9c10471c00 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:49.969841 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ccece-64d8-41e5-ab10-b522388126f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:49.970493 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ccece-64d8-41e5-ab10-b522388126f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:49.970806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ccece-64d8-41e5-ab10-b522388126f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:49.971087 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ccece-64d8-41e5-ab10-b522388126f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:49.971763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ccece-64d8-41e5-ab10-b522388126f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:49.973751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ccece-64d8-41e5-ab10-b522388126f6 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:49.995835 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1230/2472] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:49 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:50.000699 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059601b3-2c47-49ad-8680-34325123217f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:50.011276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059601b3-2c47-49ad-8680-34325123217f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:50.016137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affec3c9-87b6-48a5-af08-106eb47bd5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:50.016758 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affec3c9-87b6-48a5-af08-106eb47bd5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:50.017047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affec3c9-87b6-48a5-af08-106eb47bd5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:50.017312 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affec3c9-87b6-48a5-af08-106eb47bd5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:50.017950 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affec3c9-87b6-48a5-af08-106eb47bd5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:50.019568 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affec3c9-87b6-48a5-af08-106eb47bd5e6 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:50.029546 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1243/2473] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:50.033568 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ccece-64d8-41e5-ab10-b522388126f6 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:50.045155 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ccece-64d8-41e5-ab10-b522388126f6 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:50.050969 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760540d8-2e93-4f60-8537-60fa27884269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:50.051605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760540d8-2e93-4f60-8537-60fa27884269 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:50.051872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760540d8-2e93-4f60-8537-60fa27884269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:50.052136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760540d8-2e93-4f60-8537-60fa27884269 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:50.052779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760540d8-2e93-4f60-8537-60fa27884269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:50.054398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760540d8-2e93-4f60-8537-60fa27884269 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:50.066834 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1231/2474] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:50.071740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affec3c9-87b6-48a5-af08-106eb47bd5e6 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:50.082410 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affec3c9-87b6-48a5-af08-106eb47bd5e6 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:50.087485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63210f-7426-4827-93b2-a9bbfce58987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:50.088144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63210f-7426-4827-93b2-a9bbfce58987 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:50.088449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63210f-7426-4827-93b2-a9bbfce58987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:50.088760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63210f-7426-4827-93b2-a9bbfce58987 admin admin] PATH_INFO: `/v3/projects/f16cd186e0ca498ea2495819e9861a84/users/1010ffbe1d5e4dbf8bff1d351ae2898c/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:50.103281 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d63210f-7426-4827-93b2-a9bbfce58987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f16cd186e0ca498ea2495819e9861a84, user_id=1010ffbe1d5e4dbf8bff1d351ae2898c, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:50.105930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d63210f-7426-4827-93b2-a9bbfce58987 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:50.120341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1244/2475] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:50 2022] PUT /identity/v3/projects/f16cd186e0ca498ea2495819e9861a84/users/1010ffbe1d5e4dbf8bff1d351ae2898c/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:50.131424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854db244-a9a5-4e38-918a-935662f2140d None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:50.131712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854db244-a9a5-4e38-918a-935662f2140d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:50.131988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854db244-a9a5-4e38-918a-935662f2140d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:50.160498 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-854db244-a9a5-4e38-918a-935662f2140d None None] MFA Rules not processed for user `1010ffbe1d5e4dbf8bff1d351ae2898c`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:50.161783 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-854db244-a9a5-4e38-918a-935662f2140d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:50.175810 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:50.233484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1232/2476] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:14:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:50.245223 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63210f-7426-4827-93b2-a9bbfce58987 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:50.246955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9d63210f-7426-4827-93b2-a9bbfce58987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:50.251278 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:50.328316 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63210f-7426-4827-93b2-a9bbfce58987 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:50.332062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77197a-11ca-4db9-a984-8ad7bbaa6b1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:50.332666 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77197a-11ca-4db9-a984-8ad7bbaa6b1d service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:50.332944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77197a-11ca-4db9-a984-8ad7bbaa6b1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:50.333216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77197a-11ca-4db9-a984-8ad7bbaa6b1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:50.341353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77197a-11ca-4db9-a984-8ad7bbaa6b1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:50.342791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77197a-11ca-4db9-a984-8ad7bbaa6b1d service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:50.352197 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1245/2477] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:59.369600 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edd363-7193-482c-b043-7d7e6e58577d None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:59.370170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edd363-7193-482c-b043-7d7e6e58577d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:59.370170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edd363-7193-482c-b043-7d7e6e58577d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:59.400484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-55edd363-7193-482c-b043-7d7e6e58577d None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:59.401696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-55edd363-7193-482c-b043-7d7e6e58577d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:59.463252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1233/2478] 10.222.0.62 () {58 vars in 1009 bytes} [Fri Sep 9 12:14:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:59.512325 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77197a-11ca-4db9-a984-8ad7bbaa6b1d service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:59.513759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fe77197a-11ca-4db9-a984-8ad7bbaa6b1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:59.589835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77197a-11ca-4db9-a984-8ad7bbaa6b1d service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:59.595704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d262f0-0b50-4e1d-a408-8295c51202b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:59.596348 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d262f0-0b50-4e1d-a408-8295c51202b9 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:59.596767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d262f0-0b50-4e1d-a408-8295c51202b9 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:59.597014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d262f0-0b50-4e1d-a408-8295c51202b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:59.605407 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d262f0-0b50-4e1d-a408-8295c51202b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:59.607080 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d262f0-0b50-4e1d-a408-8295c51202b9 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:59.615158 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1246/2479] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:14:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:15:00.349544 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edd363-7193-482c-b043-7d7e6e58577d None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:00.433695 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edd363-7193-482c-b043-7d7e6e58577d None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:00.439777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c2f09-9c26-4487-b206-990ea0ee4606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:00.440740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c2f09-9c26-4487-b206-990ea0ee4606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:00.441099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c2f09-9c26-4487-b206-990ea0ee4606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:00.441428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c2f09-9c26-4487-b206-990ea0ee4606 admin admin] PATH_INFO: `/v3/users/1010ffbe1d5e4dbf8bff1d351ae2898c` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:00.442985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c2f09-9c26-4487-b206-990ea0ee4606 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1010ffbe1d5e4dbf8bff1d351ae2898c)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:00.444932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c2f09-9c26-4487-b206-990ea0ee4606 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:00.510047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7a9c2f09-9c26-4487-b206-990ea0ee4606 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1010ffbe1d5e4dbf8bff1d351ae2898c'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:00.512016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:00.522145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7a9c2f09-9c26-4487-b206-990ea0ee4606 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1010ffbe1d5e4dbf8bff1d351ae2898c'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:00.539303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7a9c2f09-9c26-4487-b206-990ea0ee4606 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1010ffbe1d5e4dbf8bff1d351ae2898c'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:00.548598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7a9c2f09-9c26-4487-b206-990ea0ee4606 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1010ffbe1d5e4dbf8bff1d351ae2898c'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:00.549771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7a9c2f09-9c26-4487-b206-990ea0ee4606 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1010ffbe1d5e4dbf8bff1d351ae2898c'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:00.557997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1234/2480] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:15:00 2022] DELETE /identity/v3/users/1010ffbe1d5e4dbf8bff1d351ae2898c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:00.562041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a001d7-733b-4cd1-bc2c-bc9c23f9fea7 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:00.562625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a001d7-733b-4cd1-bc2c-bc9c23f9fea7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:00.562625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a001d7-733b-4cd1-bc2c-bc9c23f9fea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:00.588523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-21a001d7-733b-4cd1-bc2c-bc9c23f9fea7 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:00.590039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-21a001d7-733b-4cd1-bc2c-bc9c23f9fea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:00.676176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1247/2481] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:00.684229 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c2f09-9c26-4487-b206-990ea0ee4606 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:00.685698 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7a9c2f09-9c26-4487-b206-990ea0ee4606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:00.752157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c2f09-9c26-4487-b206-990ea0ee4606 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:00.756092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1f2dc-2347-4570-8b6c-e4cc7b37ab82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:00.756707 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1f2dc-2347-4570-8b6c-e4cc7b37ab82 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:00.756984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1f2dc-2347-4570-8b6c-e4cc7b37ab82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:00.757384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1f2dc-2347-4570-8b6c-e4cc7b37ab82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:00.769158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1f2dc-2347-4570-8b6c-e4cc7b37ab82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:00.770666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1f2dc-2347-4570-8b6c-e4cc7b37ab82 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:00.790383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1235/2482] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:01.089460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a001d7-733b-4cd1-bc2c-bc9c23f9fea7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:01.091925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-21a001d7-733b-4cd1-bc2c-bc9c23f9fea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:01.107743 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a001d7-733b-4cd1-bc2c-bc9c23f9fea7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:01.113481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1eab9-b530-45e9-9418-e1c075257b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:01.113817 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1eab9-b530-45e9-9418-e1c075257b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:01.115051 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1eab9-b530-45e9-9418-e1c075257b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:01.115190 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1eab9-b530-45e9-9418-e1c075257b8b admin admin] PATH_INFO: `/v3/projects/f16cd186e0ca498ea2495819e9861a84` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:01.116625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1eab9-b530-45e9-9418-e1c075257b8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f16cd186e0ca498ea2495819e9861a84)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:01.118903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1eab9-b530-45e9-9418-e1c075257b8b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:01.135105 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:01.179949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e7c1eab9-b530-45e9-9418-e1c075257b8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f16cd186e0ca498ea2495819e9861a84', 'request_id': 'req-e7c1eab9-b530-45e9-9418-e1c075257b8b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.189740 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e7c1eab9-b530-45e9-9418-e1c075257b8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f16cd186e0ca498ea2495819e9861a84', 'request_id': 'req-e7c1eab9-b530-45e9-9418-e1c075257b8b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.202083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e7c1eab9-b530-45e9-9418-e1c075257b8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f16cd186e0ca498ea2495819e9861a84', 'request_id': 'req-e7c1eab9-b530-45e9-9418-e1c075257b8b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.215336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e7c1eab9-b530-45e9-9418-e1c075257b8b admin admin] The token cache is being invalidate because project f16cd186e0ca498ea2495819e9861a84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:15:01.215681 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e7c1eab9-b530-45e9-9418-e1c075257b8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.216448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e7c1eab9-b530-45e9-9418-e1c075257b8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.218389 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1248/2483] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:15:01 2022] DELETE /identity/v3/projects/f16cd186e0ca498ea2495819e9861a84 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:01.223909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3ee09-1227-4311-ab54-fa97309e5f4c None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:01.224098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3ee09-1227-4311-ab54-fa97309e5f4c None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:01.224322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3ee09-1227-4311-ab54-fa97309e5f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:01.245833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d3b3ee09-1227-4311-ab54-fa97309e5f4c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:01.247159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d3b3ee09-1227-4311-ab54-fa97309e5f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:01.329361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1236/2484] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:01.338082 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eae480-f29b-49c6-81d8-400f5ffa3966 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:01.338390 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eae480-f29b-49c6-81d8-400f5ffa3966 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:01.338663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eae480-f29b-49c6-81d8-400f5ffa3966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:01.362038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b4eae480-f29b-49c6-81d8-400f5ffa3966 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:01.363437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b4eae480-f29b-49c6-81d8-400f5ffa3966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:01.371837 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1249/2485] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:01.376381 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3ee09-1227-4311-ab54-fa97309e5f4c None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:01.392500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3ee09-1227-4311-ab54-fa97309e5f4c None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:01.398531 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e16d1-5e59-46cb-aa50-6c36ed637620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:01.399298 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e16d1-5e59-46cb-aa50-6c36ed637620 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:01.399632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e16d1-5e59-46cb-aa50-6c36ed637620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:01.400095 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e16d1-5e59-46cb-aa50-6c36ed637620 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:01.400640 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e16d1-5e59-46cb-aa50-6c36ed637620 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:15:01.400983 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e16d1-5e59-46cb-aa50-6c36ed637620 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:01.402763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e16d1-5e59-46cb-aa50-6c36ed637620 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:01.418174 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1237/2486] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:15:01.420728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eae480-f29b-49c6-81d8-400f5ffa3966 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:01.431532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eae480-f29b-49c6-81d8-400f5ffa3966 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:01.437886 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111a991-2de5-4733-8fcb-0eaaffc1ed3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:01.438760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111a991-2de5-4733-8fcb-0eaaffc1ed3d admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:01.439181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111a991-2de5-4733-8fcb-0eaaffc1ed3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:01.439323 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111a991-2de5-4733-8fcb-0eaaffc1ed3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:01.440227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111a991-2de5-4733-8fcb-0eaaffc1ed3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:01.442454 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111a991-2de5-4733-8fcb-0eaaffc1ed3d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:01.477554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e16d1-5e59-46cb-aa50-6c36ed637620 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:01.478580 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1250/2487] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:15:01 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:01.490368 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e16d1-5e59-46cb-aa50-6c36ed637620 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:01.554921 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:01.569867 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79448e0-e559-40ef-82f2-f6ce38bdee4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:01.570519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79448e0-e559-40ef-82f2-f6ce38bdee4b admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:01.570917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79448e0-e559-40ef-82f2-f6ce38bdee4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:01.571051 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79448e0-e559-40ef-82f2-f6ce38bdee4b admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:01.571679 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79448e0-e559-40ef-82f2-f6ce38bdee4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:01.573691 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79448e0-e559-40ef-82f2-f6ce38bdee4b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:01.599147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1238/2488] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:15:01 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:15:01.602669 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111a991-2de5-4733-8fcb-0eaaffc1ed3d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:01.614474 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111a991-2de5-4733-8fcb-0eaaffc1ed3d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:01.625135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f490bc-5f64-4c64-81c4-82a8a85e4023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:01.627059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f490bc-5f64-4c64-81c4-82a8a85e4023 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:01.627590 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f490bc-5f64-4c64-81c4-82a8a85e4023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:01.628141 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f490bc-5f64-4c64-81c4-82a8a85e4023 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:01.629089 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f490bc-5f64-4c64-81c4-82a8a85e4023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:01.630921 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f490bc-5f64-4c64-81c4-82a8a85e4023 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:01.643230 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1251/2489] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:01.647305 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79448e0-e559-40ef-82f2-f6ce38bdee4b admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:01.659767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79448e0-e559-40ef-82f2-f6ce38bdee4b admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:01.672427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9737f-95bd-4f78-9bfb-9e3f38e6f4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:01.673058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9737f-95bd-4f78-9bfb-9e3f38e6f4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:01.673314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9737f-95bd-4f78-9bfb-9e3f38e6f4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:01.673580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9737f-95bd-4f78-9bfb-9e3f38e6f4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:01.674264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9737f-95bd-4f78-9bfb-9e3f38e6f4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:01.675873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9737f-95bd-4f78-9bfb-9e3f38e6f4e1 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:01.694159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1239/2490] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:01.695979 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f490bc-5f64-4c64-81c4-82a8a85e4023 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:01.710214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f490bc-5f64-4c64-81c4-82a8a85e4023 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:01.721152 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd481c-3397-47ee-80b7-cf449aac7e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:01.721993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd481c-3397-47ee-80b7-cf449aac7e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:01.722435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd481c-3397-47ee-80b7-cf449aac7e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:01.722807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd481c-3397-47ee-80b7-cf449aac7e4c admin admin] PATH_INFO: `/v3/projects/68a3919f2172442fa280c380a7bff271/users/cb72d25947724d028d948292925258d9/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:01.739448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd481c-3397-47ee-80b7-cf449aac7e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=68a3919f2172442fa280c380a7bff271, user_id=cb72d25947724d028d948292925258d9, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:01.744951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd481c-3397-47ee-80b7-cf449aac7e4c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:01.770170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1252/2491] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:15:01 2022] PUT /identity/v3/projects/68a3919f2172442fa280c380a7bff271/users/cb72d25947724d028d948292925258d9/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:01.772498 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e6fac-b34d-4de2-a835-0067bc64aa3f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:01.772992 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e6fac-b34d-4de2-a835-0067bc64aa3f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:01.773181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e6fac-b34d-4de2-a835-0067bc64aa3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:01.801772 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-2b8e6fac-b34d-4de2-a835-0067bc64aa3f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:01.803528 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2b8e6fac-b34d-4de2-a835-0067bc64aa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:01.890173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1240/2492] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:01.898198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd481c-3397-47ee-80b7-cf449aac7e4c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:01.900089 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a9cd481c-3397-47ee-80b7-cf449aac7e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:01.969389 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd481c-3397-47ee-80b7-cf449aac7e4c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:01.976821 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e50421-9de0-4366-af78-f8890124bd77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:01.977445 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e50421-9de0-4366-af78-f8890124bd77 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:01.977723 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e50421-9de0-4366-af78-f8890124bd77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:01.977995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e50421-9de0-4366-af78-f8890124bd77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:01.990580 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e50421-9de0-4366-af78-f8890124bd77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:01.992397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e50421-9de0-4366-af78-f8890124bd77 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:02.005870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1253/2493] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:07.564589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e57880-67af-44f1-aec0-68739c003c84 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:07.565091 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e57880-67af-44f1-aec0-68739c003c84 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:07.565205 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e57880-67af-44f1-aec0-68739c003c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:07.601359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-29e57880-67af-44f1-aec0-68739c003c84 None None] MFA Rules not processed for user `cb72d25947724d028d948292925258d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:07.602726 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-29e57880-67af-44f1-aec0-68739c003c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:07.713318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1241/2494] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:15:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:07.720683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e50421-9de0-4366-af78-f8890124bd77 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:07.732448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e50421-9de0-4366-af78-f8890124bd77 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:07.743800 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98490aa0-53d1-478a-81e5-c1810e4725c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:07.744812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98490aa0-53d1-478a-81e5-c1810e4725c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:07.745137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98490aa0-53d1-478a-81e5-c1810e4725c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:07.745633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98490aa0-53d1-478a-81e5-c1810e4725c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:07.752992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98490aa0-53d1-478a-81e5-c1810e4725c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:07.755146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98490aa0-53d1-478a-81e5-c1810e4725c4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:07.793205 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1254/2495] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:15:07 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:07.798978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e57880-67af-44f1-aec0-68739c003c84 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:07.816758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e57880-67af-44f1-aec0-68739c003c84 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:07.904483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172039d-d0bf-4f98-999a-7b5030026c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:07.907500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172039d-d0bf-4f98-999a-7b5030026c7d admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:07.907780 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172039d-d0bf-4f98-999a-7b5030026c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:07.908051 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172039d-d0bf-4f98-999a-7b5030026c7d admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:07.908835 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172039d-d0bf-4f98-999a-7b5030026c7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:07.912133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172039d-d0bf-4f98-999a-7b5030026c7d admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:07.928067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:07.946048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1242/2496] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:15:07 2022] POST /identity/v3/users => generated 299 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:07.950927 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98490aa0-53d1-478a-81e5-c1810e4725c4 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:07.965655 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98490aa0-53d1-478a-81e5-c1810e4725c4 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:07.973969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6eca3-16a0-4e4d-aba5-2fb45fff4bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:07.975144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6eca3-16a0-4e4d-aba5-2fb45fff4bab admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:07.975494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6eca3-16a0-4e4d-aba5-2fb45fff4bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:07.975827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6eca3-16a0-4e4d-aba5-2fb45fff4bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:07.976594 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6eca3-16a0-4e4d-aba5-2fb45fff4bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:07.978292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6eca3-16a0-4e4d-aba5-2fb45fff4bab admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:07.991823 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1255/2497] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:07.997676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172039d-d0bf-4f98-999a-7b5030026c7d admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:08.013263 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172039d-d0bf-4f98-999a-7b5030026c7d admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:08.020130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e28d44-8348-44d5-801c-2acebac360e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:08.020815 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e28d44-8348-44d5-801c-2acebac360e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:08.021094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e28d44-8348-44d5-801c-2acebac360e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:08.021366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e28d44-8348-44d5-801c-2acebac360e2 admin admin] PATH_INFO: `/v3/projects/4e7db0beb2fa4e4a96416bb3673763f3/users/f819395e7004410f9c0cd83bd7f25e4d/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:08.041848 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e28d44-8348-44d5-801c-2acebac360e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e7db0beb2fa4e4a96416bb3673763f3, user_id=f819395e7004410f9c0cd83bd7f25e4d, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:08.045066 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e28d44-8348-44d5-801c-2acebac360e2 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:08.074076 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6eca3-16a0-4e4d-aba5-2fb45fff4bab admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:08.074575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1243/2498] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:15:07 2022] PUT /identity/v3/projects/4e7db0beb2fa4e4a96416bb3673763f3/users/f819395e7004410f9c0cd83bd7f25e4d/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:08.081407 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:08.086763 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6eca3-16a0-4e4d-aba5-2fb45fff4bab admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:08.172021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6defb97-36d1-419c-8639-3fdb1ab44f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:08.172699 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6defb97-36d1-419c-8639-3fdb1ab44f75 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:08.172976 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6defb97-36d1-419c-8639-3fdb1ab44f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:08.173244 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6defb97-36d1-419c-8639-3fdb1ab44f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:08.173930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6defb97-36d1-419c-8639-3fdb1ab44f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:08.175622 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6defb97-36d1-419c-8639-3fdb1ab44f75 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:08.204539 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1256/2499] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:08.206001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e28d44-8348-44d5-801c-2acebac360e2 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:08.221428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e28d44-8348-44d5-801c-2acebac360e2 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:08.229326 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e3d6c-c0c1-42f8-a1f8-50ef93b5ace5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:08.230080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e3d6c-c0c1-42f8-a1f8-50ef93b5ace5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:08.230401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e3d6c-c0c1-42f8-a1f8-50ef93b5ace5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:08.230678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e3d6c-c0c1-42f8-a1f8-50ef93b5ace5 admin admin] PATH_INFO: `/v3/projects/4e7db0beb2fa4e4a96416bb3673763f3/users/f819395e7004410f9c0cd83bd7f25e4d/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:08.233994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e3d6c-c0c1-42f8-a1f8-50ef93b5ace5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e7db0beb2fa4e4a96416bb3673763f3, user_id=f819395e7004410f9c0cd83bd7f25e4d, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:08.239897 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e3d6c-c0c1-42f8-a1f8-50ef93b5ace5 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:08.300673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1244/2500] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:15:08 2022] PUT /identity/v3/projects/4e7db0beb2fa4e4a96416bb3673763f3/users/f819395e7004410f9c0cd83bd7f25e4d/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:14.200227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab51f9-3c3f-4d2d-9eec-5e2beceb8159 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:14.200734 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab51f9-3c3f-4d2d-9eec-5e2beceb8159 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:14.200818 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab51f9-3c3f-4d2d-9eec-5e2beceb8159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:14.241383 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-07ab51f9-3c3f-4d2d-9eec-5e2beceb8159 None None] MFA Rules not processed for user `f819395e7004410f9c0cd83bd7f25e4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:14.242655 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-07ab51f9-3c3f-4d2d-9eec-5e2beceb8159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:14.335182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1257/2501] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:15:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:14.349079 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e3d6c-c0c1-42f8-a1f8-50ef93b5ace5 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:14.353046 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-965e3d6c-c0c1-42f8-a1f8-50ef93b5ace5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:14.420431 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e3d6c-c0c1-42f8-a1f8-50ef93b5ace5 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:14.424202 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f53d5a-a7f3-4042-b98b-c98f0531c36e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d477a49a1724b7e8a4699a5ebd04bde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:14.424821 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f53d5a-a7f3-4042-b98b-c98f0531c36e service glance] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:14.425099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f53d5a-a7f3-4042-b98b-c98f0531c36e service glance] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:14.425368 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f53d5a-a7f3-4042-b98b-c98f0531c36e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:14.491582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f53d5a-a7f3-4042-b98b-c98f0531c36e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:14.493180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f53d5a-a7f3-4042-b98b-c98f0531c36e service glance] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:14.503891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1245/2502] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:14.704930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d9121-0f1c-48c5-bbb0-bce2c5c07d6b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:14.705516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d9121-0f1c-48c5-bbb0-bce2c5c07d6b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:14.706010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d9121-0f1c-48c5-bbb0-bce2c5c07d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:14.734857 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-673d9121-0f1c-48c5-bbb0-bce2c5c07d6b None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:14.736093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-673d9121-0f1c-48c5-bbb0-bce2c5c07d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:14.822092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1258/2503] 10.222.0.62 () {58 vars in 1009 bytes} [Fri Sep 9 12:15:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:14.837508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f53d5a-a7f3-4042-b98b-c98f0531c36e service glance] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:14.839301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e7f53d5a-a7f3-4042-b98b-c98f0531c36e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:14.923203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f53d5a-a7f3-4042-b98b-c98f0531c36e service glance] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:14.929741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eace36-aff8-40a6-bd7a-db87ba792c68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:14.930406 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eace36-aff8-40a6-bd7a-db87ba792c68 service swift] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:14.930790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eace36-aff8-40a6-bd7a-db87ba792c68 service swift] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:14.930955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eace36-aff8-40a6-bd7a-db87ba792c68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:14.940373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eace36-aff8-40a6-bd7a-db87ba792c68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:14.942079 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eace36-aff8-40a6-bd7a-db87ba792c68 service swift] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:14.948005 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:14.953249 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1246/2504] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:15:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:15:20.419920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc88ca2-e4bd-4211-b69f-af416be25ec5 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:20.420652 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc88ca2-e4bd-4211-b69f-af416be25ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:20.420923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc88ca2-e4bd-4211-b69f-af416be25ec5 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:20.422797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1259/2505] 10.222.0.62 () {54 vars in 921 bytes} [Fri Sep 9 12:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:15:20.430324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d56e20-94e7-471a-8391-121f045b08db None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:20.431791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d56e20-94e7-471a-8391-121f045b08db None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:20.431791 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d56e20-94e7-471a-8391-121f045b08db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:20.510328 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-93d56e20-94e7-471a-8391-121f045b08db None None] MFA Rules not processed for user `1b30e80c421245b5a1ed5849b28ea90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:20.511729 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-93d56e20-94e7-471a-8391-121f045b08db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:20.623173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1247/2506] 10.222.0.62 () {58 vars in 1018 bytes} [Fri Sep 9 12:15:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:20.636412 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc88ca2-e4bd-4211-b69f-af416be25ec5 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:20.659658 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc88ca2-e4bd-4211-b69f-af416be25ec5 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:20.664233 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8d045-3043-4058-8cf9-035e8810f82e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d477a49a1724b7e8a4699a5ebd04bde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:20.664980 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8d045-3043-4058-8cf9-035e8810f82e service glance] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:20.664980 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8d045-3043-4058-8cf9-035e8810f82e service glance] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:20.665248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8d045-3043-4058-8cf9-035e8810f82e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:20.677827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8d045-3043-4058-8cf9-035e8810f82e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:20.679308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8d045-3043-4058-8cf9-035e8810f82e service glance] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:20.692424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1260/2507] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:20.782508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84230f36-857b-4e3a-b1dd-f12c0280f1b3 None None] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:20.782922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84230f36-857b-4e3a-b1dd-f12c0280f1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:20.783322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84230f36-857b-4e3a-b1dd-f12c0280f1b3 None None] PATH_INFO: `/` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:20.785262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1248/2508] 10.222.0.62 () {54 vars in 921 bytes} [Fri Sep 9 12:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:15:20.790503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51966c3-195d-4c9d-bf30-d7621c5e2329 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:20.790781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51966c3-195d-4c9d-bf30-d7621c5e2329 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:20.791335 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51966c3-195d-4c9d-bf30-d7621c5e2329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:20.823229 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d51966c3-195d-4c9d-bf30-d7621c5e2329 None None] MFA Rules not processed for user `d00634cdd3fb41e794d3af0b88ba7338`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:20.824640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d51966c3-195d-4c9d-bf30-d7621c5e2329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:20.843234 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:20.923111 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1261/2509] 10.222.0.62 () {58 vars in 1018 bytes} [Fri Sep 9 12:15:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:15:20.941092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84230f36-857b-4e3a-b1dd-f12c0280f1b3 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:20.943657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-84230f36-857b-4e3a-b1dd-f12c0280f1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:20.960599 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84230f36-857b-4e3a-b1dd-f12c0280f1b3 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:20.964679 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce1177-06a0-4e11-89b3-454b66c7af31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:20.965293 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce1177-06a0-4e11-89b3-454b66c7af31 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:20.965571 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce1177-06a0-4e11-89b3-454b66c7af31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:20.965843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce1177-06a0-4e11-89b3-454b66c7af31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:20.973846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce1177-06a0-4e11-89b3-454b66c7af31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:20.975532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce1177-06a0-4e11-89b3-454b66c7af31 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:21.000661 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1249/2510] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:21.088161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4b33c-8968-4652-bbce-d2d9b06dc3d6 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:21.088794 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4b33c-8968-4652-bbce-d2d9b06dc3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:21.089338 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4b33c-8968-4652-bbce-d2d9b06dc3d6 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:21.091259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1262/2511] 10.222.0.62 () {54 vars in 921 bytes} [Fri Sep 9 12:15:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:15:21.096972 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e62bba-d34e-4f92-b9f1-7debd5f0a355 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:21.097261 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e62bba-d34e-4f92-b9f1-7debd5f0a355 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:21.097541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e62bba-d34e-4f92-b9f1-7debd5f0a355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:21.120406 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-c8e62bba-d34e-4f92-b9f1-7debd5f0a355 None None] MFA Rules not processed for user `1b30e80c421245b5a1ed5849b28ea90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:21.122233 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-c8e62bba-d34e-4f92-b9f1-7debd5f0a355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:21.135187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1250/2512] 10.222.0.62 () {58 vars in 1018 bytes} [Fri Sep 9 12:15:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:21.148377 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4b33c-8968-4652-bbce-d2d9b06dc3d6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:21.163808 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4b33c-8968-4652-bbce-d2d9b06dc3d6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:21.167556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2841029-ddf1-4c2d-b896-b80eed79a30a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:21.168165 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2841029-ddf1-4c2d-b896-b80eed79a30a service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:21.168442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2841029-ddf1-4c2d-b896-b80eed79a30a service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:21.168717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2841029-ddf1-4c2d-b896-b80eed79a30a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:21.180497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2841029-ddf1-4c2d-b896-b80eed79a30a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:21.182273 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2841029-ddf1-4c2d-b896-b80eed79a30a service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:21.205811 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1263/2513] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:23.313162 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dba006-7506-428d-a132-78120f042cc5 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:23.313891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dba006-7506-428d-a132-78120f042cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:23.314000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dba006-7506-428d-a132-78120f042cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:23.339446 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-08dba006-7506-428d-a132-78120f042cc5 None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:23.340684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-08dba006-7506-428d-a132-78120f042cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:23.354513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1251/2514] 10.222.0.62 () {58 vars in 1009 bytes} [Fri Sep 9 12:15:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:23.368028 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2841029-ddf1-4c2d-b896-b80eed79a30a service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:23.387981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2841029-ddf1-4c2d-b896-b80eed79a30a service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:23.394908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568dbe4-9174-4e83-8d1a-b9ea93a74fdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:23.395530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568dbe4-9174-4e83-8d1a-b9ea93a74fdf service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:23.395804 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568dbe4-9174-4e83-8d1a-b9ea93a74fdf service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:23.396077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568dbe4-9174-4e83-8d1a-b9ea93a74fdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:23.406440 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568dbe4-9174-4e83-8d1a-b9ea93a74fdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:23.408084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568dbe4-9174-4e83-8d1a-b9ea93a74fdf service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:23.417768 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1264/2515] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:15:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:15:25.829933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220917c2-bddc-4990-a0db-65b8653e51c4 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:25.830879 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220917c2-bddc-4990-a0db-65b8653e51c4 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:25.830879 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220917c2-bddc-4990-a0db-65b8653e51c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:25.852412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-220917c2-bddc-4990-a0db-65b8653e51c4 None None] MFA Rules not processed for user `cb72d25947724d028d948292925258d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:25.853779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-220917c2-bddc-4990-a0db-65b8653e51c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:25.862579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1252/2516] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:33.414435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=27 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:35.845478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=55 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:39.380561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f021b-0017-49b0-8994-06d0fac46d2f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:39.381533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f021b-0017-49b0-8994-06d0fac46d2f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:39.381817 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f021b-0017-49b0-8994-06d0fac46d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:39.402389 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ec0f021b-0017-49b0-8994-06d0fac46d2f None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:39.403614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ec0f021b-0017-49b0-8994-06d0fac46d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:39.411436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1265/2517] 10.222.0.62 () {58 vars in 1009 bytes} [Fri Sep 9 12:15:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:39.424274 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220917c2-bddc-4990-a0db-65b8653e51c4 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:39.436126 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220917c2-bddc-4990-a0db-65b8653e51c4 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:39.441325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee174d-9d07-4c6c-9b3b-788673d06a8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:39.442061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee174d-9d07-4c6c-9b3b-788673d06a8e service swift] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:39.442428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee174d-9d07-4c6c-9b3b-788673d06a8e service swift] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:39.442765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee174d-9d07-4c6c-9b3b-788673d06a8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:39.451603 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee174d-9d07-4c6c-9b3b-788673d06a8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:39.453322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee174d-9d07-4c6c-9b3b-788673d06a8e service swift] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:39.461674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1253/2518] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:15:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:15:49.397428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:49.458336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=6 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:54.098560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75157514-4e82-476f-9058-9e3189cd47a2 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:54.099067 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75157514-4e82-476f-9058-9e3189cd47a2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:54.099176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75157514-4e82-476f-9058-9e3189cd47a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:54.119090 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-75157514-4e82-476f-9058-9e3189cd47a2 None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:54.120259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-75157514-4e82-476f-9058-9e3189cd47a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:54.128044 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1266/2519] 10.222.0.62 () {58 vars in 967 bytes} [Fri Sep 9 12:15:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:54.141358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee174d-9d07-4c6c-9b3b-788673d06a8e service swift] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:54.152042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee174d-9d07-4c6c-9b3b-788673d06a8e service swift] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:54.157441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1e4c1-8ee4-44aa-81a5-64b31ea18817 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:54.158040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1e4c1-8ee4-44aa-81a5-64b31ea18817 service swift] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:54.158344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1e4c1-8ee4-44aa-81a5-64b31ea18817 service swift] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:54.158615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1e4c1-8ee4-44aa-81a5-64b31ea18817 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:54.167154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1e4c1-8ee4-44aa-81a5-64b31ea18817 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:54.168794 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1e4c1-8ee4-44aa-81a5-64b31ea18817 service swift] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:54.177299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1254/2520] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:15:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:16:04.113760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:04.174080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=6 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:09.870453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75157514-4e82-476f-9058-9e3189cd47a2 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:09.881224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75157514-4e82-476f-9058-9e3189cd47a2 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:09.888672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc7dac-a52a-40cf-97e0-bd82a9430772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:09.889397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc7dac-a52a-40cf-97e0-bd82a9430772 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:09.889747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc7dac-a52a-40cf-97e0-bd82a9430772 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:09.890017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc7dac-a52a-40cf-97e0-bd82a9430772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:09.891810 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-24dc7dac-a52a-40cf-97e0-bd82a9430772 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:09.907303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc7dac-a52a-40cf-97e0-bd82a9430772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:09.908968 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc7dac-a52a-40cf-97e0-bd82a9430772 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:09.920204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1267/2521] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:14.299462 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1e4c1-8ee4-44aa-81a5-64b31ea18817 service swift] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:14.309160 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1e4c1-8ee4-44aa-81a5-64b31ea18817 service swift] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:14.312829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649bd6b-709d-48de-9236-3d73966d74c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:14.313416 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649bd6b-709d-48de-9236-3d73966d74c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:14.313696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649bd6b-709d-48de-9236-3d73966d74c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:14.313967 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649bd6b-709d-48de-9236-3d73966d74c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:14.315762 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6649bd6b-709d-48de-9236-3d73966d74c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:14.329037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649bd6b-709d-48de-9236-3d73966d74c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:14.330560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649bd6b-709d-48de-9236-3d73966d74c4 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:14.341803 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1255/2522] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:14.476278 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc7dac-a52a-40cf-97e0-bd82a9430772 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:14.484977 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc7dac-a52a-40cf-97e0-bd82a9430772 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:14.492589 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f5192-0699-4d3f-b4d8-bf94c3bb812d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:14.493712 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f5192-0699-4d3f-b4d8-bf94c3bb812d service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:14.494170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f5192-0699-4d3f-b4d8-bf94c3bb812d service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:14.494423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f5192-0699-4d3f-b4d8-bf94c3bb812d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:14.504235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f5192-0699-4d3f-b4d8-bf94c3bb812d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:14.505643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f5192-0699-4d3f-b4d8-bf94c3bb812d service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:14.515186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1268/2523] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:15.228286 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649bd6b-709d-48de-9236-3d73966d74c4 service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:15.246401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649bd6b-709d-48de-9236-3d73966d74c4 service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:15.252699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be1da4-806f-482d-ba1d-d29f20bbc6fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:15.253306 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be1da4-806f-482d-ba1d-d29f20bbc6fb service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:15.253628 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be1da4-806f-482d-ba1d-d29f20bbc6fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:15.253899 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be1da4-806f-482d-ba1d-d29f20bbc6fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:15.256086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-51be1da4-806f-482d-ba1d-d29f20bbc6fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:15.327215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be1da4-806f-482d-ba1d-d29f20bbc6fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:15.328661 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be1da4-806f-482d-ba1d-d29f20bbc6fb service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:15.338724 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1256/2524] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:20.072819 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f5192-0699-4d3f-b4d8-bf94c3bb812d service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:20.074448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-169f5192-0699-4d3f-b4d8-bf94c3bb812d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:20.153587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be1da4-806f-482d-ba1d-d29f20bbc6fb service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:20.155117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-51be1da4-806f-482d-ba1d-d29f20bbc6fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:20.156638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f5192-0699-4d3f-b4d8-bf94c3bb812d service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:20.161788 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696067af-537e-4f95-a023-1859ee9db036 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:20.162431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696067af-537e-4f95-a023-1859ee9db036 service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:20.162695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696067af-537e-4f95-a023-1859ee9db036 service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:20.162960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696067af-537e-4f95-a023-1859ee9db036 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:20.165170 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-696067af-537e-4f95-a023-1859ee9db036 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:20.168264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be1da4-806f-482d-ba1d-d29f20bbc6fb service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:20.172961 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdcfd3-9768-43be-9234-612a51024f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:20.173932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdcfd3-9768-43be-9234-612a51024f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:20.173932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdcfd3-9768-43be-9234-612a51024f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:20.174157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdcfd3-9768-43be-9234-612a51024f82 admin admin] PATH_INFO: `/v3/users/bfc29d15a7344754aa53d55379e2751f` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:20.178392 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696067af-537e-4f95-a023-1859ee9db036 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:20.179995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696067af-537e-4f95-a023-1859ee9db036 service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:20.189742 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdcfd3-9768-43be-9234-612a51024f82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc29d15a7344754aa53d55379e2751f)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:20.191504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1269/2525] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:20.192225 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdcfd3-9768-43be-9234-612a51024f82 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:20.245365 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-15fdcfd3-9768-43be-9234-612a51024f82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc29d15a7344754aa53d55379e2751f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:20.253020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-15fdcfd3-9768-43be-9234-612a51024f82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc29d15a7344754aa53d55379e2751f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:20.265559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-15fdcfd3-9768-43be-9234-612a51024f82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc29d15a7344754aa53d55379e2751f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:20.272338 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-15fdcfd3-9768-43be-9234-612a51024f82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc29d15a7344754aa53d55379e2751f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:20.273014 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-15fdcfd3-9768-43be-9234-612a51024f82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc29d15a7344754aa53d55379e2751f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:20.278846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1257/2526] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:20 2022] DELETE /identity/v3/users/bfc29d15a7344754aa53d55379e2751f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:20.460886 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696067af-537e-4f95-a023-1859ee9db036 service placement] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:20.474152 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696067af-537e-4f95-a023-1859ee9db036 service placement] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:20.537270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28de3a8-f927-40ea-9c2b-a188c957f67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:20.537904 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28de3a8-f927-40ea-9c2b-a188c957f67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:20.538218 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28de3a8-f927-40ea-9c2b-a188c957f67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:20.538494 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28de3a8-f927-40ea-9c2b-a188c957f67e admin admin] PATH_INFO: `/v3/projects/0475d5537c6f4116ac0905e90153bbc7` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:20.549033 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28de3a8-f927-40ea-9c2b-a188c957f67e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0475d5537c6f4116ac0905e90153bbc7)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:20.550962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28de3a8-f927-40ea-9c2b-a188c957f67e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:20.598899 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d28de3a8-f927-40ea-9c2b-a188c957f67e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0475d5537c6f4116ac0905e90153bbc7', 'request_id': 'req-d28de3a8-f927-40ea-9c2b-a188c957f67e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:20.608154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d28de3a8-f927-40ea-9c2b-a188c957f67e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0475d5537c6f4116ac0905e90153bbc7', 'request_id': 'req-d28de3a8-f927-40ea-9c2b-a188c957f67e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:20.609669 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:20.621784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d28de3a8-f927-40ea-9c2b-a188c957f67e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0475d5537c6f4116ac0905e90153bbc7', 'request_id': 'req-d28de3a8-f927-40ea-9c2b-a188c957f67e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:20.641465 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d28de3a8-f927-40ea-9c2b-a188c957f67e admin admin] The token cache is being invalidate because project 0475d5537c6f4116ac0905e90153bbc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:20.641752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d28de3a8-f927-40ea-9c2b-a188c957f67e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:20.642468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d28de3a8-f927-40ea-9c2b-a188c957f67e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:20.647294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1270/2527] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:20 2022] DELETE /identity/v3/projects/0475d5537c6f4116ac0905e90153bbc7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:20.649012 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2135b-8518-43c0-8cd9-bd5de15418bd None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:20.649292 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2135b-8518-43c0-8cd9-bd5de15418bd None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:20.649632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2135b-8518-43c0-8cd9-bd5de15418bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:20.674083 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-fcd2135b-8518-43c0-8cd9-bd5de15418bd None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:20.675356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-fcd2135b-8518-43c0-8cd9-bd5de15418bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:20.715799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:20.740805 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1258/2528] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:20.749493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80752abb-c8da-4d06-bbf8-e45398527f3d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:20.749782 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80752abb-c8da-4d06-bbf8-e45398527f3d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:20.750056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80752abb-c8da-4d06-bbf8-e45398527f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:20.771534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-80752abb-c8da-4d06-bbf8-e45398527f3d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:20.772795 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-80752abb-c8da-4d06-bbf8-e45398527f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:20.781096 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1271/2529] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:20.785367 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2135b-8518-43c0-8cd9-bd5de15418bd None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:20.795760 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2135b-8518-43c0-8cd9-bd5de15418bd None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:20.801036 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dca02-1572-41c8-831a-2153b7c70fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:20.801645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dca02-1572-41c8-831a-2153b7c70fbc admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:20.801922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dca02-1572-41c8-831a-2153b7c70fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:20.802223 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dca02-1572-41c8-831a-2153b7c70fbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:20.802880 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273dca02-1572-41c8-831a-2153b7c70fbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:16:20.803184 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273dca02-1572-41c8-831a-2153b7c70fbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:20.805008 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273dca02-1572-41c8-831a-2153b7c70fbc admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:20.818623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1259/2530] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:16:20.823216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80752abb-c8da-4d06-bbf8-e45398527f3d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:20.833851 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80752abb-c8da-4d06-bbf8-e45398527f3d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:20.838916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfd1b1-996c-42d8-89b6-c384ade23308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:20.839526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfd1b1-996c-42d8-89b6-c384ade23308 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:20.839799 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfd1b1-996c-42d8-89b6-c384ade23308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:20.840066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfd1b1-996c-42d8-89b6-c384ade23308 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:20.840691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dfd1b1-996c-42d8-89b6-c384ade23308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:20.842607 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dfd1b1-996c-42d8-89b6-c384ade23308 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:20.883518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1272/2531] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:20 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:20.887973 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dca02-1572-41c8-831a-2153b7c70fbc admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:20.898702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dca02-1572-41c8-831a-2153b7c70fbc admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:20.968322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba79e8c-2ac4-40f1-8b4b-f7e990fc17fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:20.968841 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba79e8c-2ac4-40f1-8b4b-f7e990fc17fc admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:20.969118 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba79e8c-2ac4-40f1-8b4b-f7e990fc17fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:20.969387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba79e8c-2ac4-40f1-8b4b-f7e990fc17fc admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:20.970025 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba79e8c-2ac4-40f1-8b4b-f7e990fc17fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:20.972463 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba79e8c-2ac4-40f1-8b4b-f7e990fc17fc admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:20.993545 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1260/2532] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:20 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:20.997684 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfd1b1-996c-42d8-89b6-c384ade23308 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:21.007909 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfd1b1-996c-42d8-89b6-c384ade23308 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:21.012575 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f146c2-72bb-4b93-ba60-25dcb0cd5b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:21.013164 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f146c2-72bb-4b93-ba60-25dcb0cd5b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:21.013436 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f146c2-72bb-4b93-ba60-25dcb0cd5b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:21.013704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f146c2-72bb-4b93-ba60-25dcb0cd5b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:21.014360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f146c2-72bb-4b93-ba60-25dcb0cd5b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:21.015944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f146c2-72bb-4b93-ba60-25dcb0cd5b1b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:21.025774 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1273/2533] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:21.029787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba79e8c-2ac4-40f1-8b4b-f7e990fc17fc admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:21.040253 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba79e8c-2ac4-40f1-8b4b-f7e990fc17fc admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:21.045156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4103c2-509f-46aa-b14a-a79d256d89a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:21.045745 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4103c2-509f-46aa-b14a-a79d256d89a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:21.046022 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4103c2-509f-46aa-b14a-a79d256d89a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:21.046324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4103c2-509f-46aa-b14a-a79d256d89a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:21.046965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4103c2-509f-46aa-b14a-a79d256d89a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:21.048639 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4103c2-509f-46aa-b14a-a79d256d89a4 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:21.059962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1261/2534] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:21.064172 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f146c2-72bb-4b93-ba60-25dcb0cd5b1b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:21.075421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f146c2-72bb-4b93-ba60-25dcb0cd5b1b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:21.080781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052b7f2-6eeb-4081-8fc3-d038a9e45f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:21.081437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052b7f2-6eeb-4081-8fc3-d038a9e45f3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:21.081719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052b7f2-6eeb-4081-8fc3-d038a9e45f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:21.081998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052b7f2-6eeb-4081-8fc3-d038a9e45f3f admin admin] PATH_INFO: `/v3/projects/b54810059a014d3d99879824cfcbb5f6/users/d5fcb31d881b45fe9c790513767bb895/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:21.097221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052b7f2-6eeb-4081-8fc3-d038a9e45f3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b54810059a014d3d99879824cfcbb5f6, user_id=d5fcb31d881b45fe9c790513767bb895, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:21.099915 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052b7f2-6eeb-4081-8fc3-d038a9e45f3f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:21.114843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1274/2535] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:21 2022] PUT /identity/v3/projects/b54810059a014d3d99879824cfcbb5f6/users/d5fcb31d881b45fe9c790513767bb895/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:21.125562 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842f579-0854-417c-8d25-53bc95c69aa9 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:21.125845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842f579-0854-417c-8d25-53bc95c69aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:21.126128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842f579-0854-417c-8d25-53bc95c69aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:21.153400 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-7842f579-0854-417c-8d25-53bc95c69aa9 None None] MFA Rules not processed for user `d5fcb31d881b45fe9c790513767bb895`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:21.154671 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7842f579-0854-417c-8d25-53bc95c69aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:21.221712 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1262/2536] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:21.233080 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052b7f2-6eeb-4081-8fc3-d038a9e45f3f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:21.234837 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c052b7f2-6eeb-4081-8fc3-d038a9e45f3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:21.307930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052b7f2-6eeb-4081-8fc3-d038a9e45f3f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:21.313431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b234f90-f160-42f1-a9d2-44af39868edb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:21.314037 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b234f90-f160-42f1-a9d2-44af39868edb service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:21.314367 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b234f90-f160-42f1-a9d2-44af39868edb service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:21.314641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b234f90-f160-42f1-a9d2-44af39868edb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:21.322452 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b234f90-f160-42f1-a9d2-44af39868edb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:21.324027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b234f90-f160-42f1-a9d2-44af39868edb service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:21.333239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1275/2537] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:24.014684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842f579-0854-417c-8d25-53bc95c69aa9 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:24.054290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:24.079101 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842f579-0854-417c-8d25-53bc95c69aa9 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:24.083638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1e0fa-0439-494c-9f45-8b017894b6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:24.084276 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1e0fa-0439-494c-9f45-8b017894b6ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:24.084557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1e0fa-0439-494c-9f45-8b017894b6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:24.084858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1e0fa-0439-494c-9f45-8b017894b6ec admin admin] PATH_INFO: `/v3/users/d5fcb31d881b45fe9c790513767bb895` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:24.086024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c1e0fa-0439-494c-9f45-8b017894b6ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5fcb31d881b45fe9c790513767bb895)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:24.088024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c1e0fa-0439-494c-9f45-8b017894b6ec admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:24.137181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-66c1e0fa-0439-494c-9f45-8b017894b6ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5fcb31d881b45fe9c790513767bb895'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:24.144383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-66c1e0fa-0439-494c-9f45-8b017894b6ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5fcb31d881b45fe9c790513767bb895'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:24.157612 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-66c1e0fa-0439-494c-9f45-8b017894b6ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5fcb31d881b45fe9c790513767bb895'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:24.166041 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-66c1e0fa-0439-494c-9f45-8b017894b6ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5fcb31d881b45fe9c790513767bb895'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:24.166744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-66c1e0fa-0439-494c-9f45-8b017894b6ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5fcb31d881b45fe9c790513767bb895'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:24.174383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1263/2538] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:24 2022] DELETE /identity/v3/users/d5fcb31d881b45fe9c790513767bb895 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:24.178882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563247b-e09b-4df7-b29f-c152b51918a9 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:24.179422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563247b-e09b-4df7-b29f-c152b51918a9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:24.179422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563247b-e09b-4df7-b29f-c152b51918a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:24.200151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4563247b-e09b-4df7-b29f-c152b51918a9 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:24.201359 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4563247b-e09b-4df7-b29f-c152b51918a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:24.261639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:24.272784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1276/2539] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:24.282768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1e0fa-0439-494c-9f45-8b017894b6ec admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:24.284200 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-66c1e0fa-0439-494c-9f45-8b017894b6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:24.342247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1e0fa-0439-494c-9f45-8b017894b6ec admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:24.345912 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f458ce-9d3f-43c5-a130-9757b588c6c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:24.346571 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f458ce-9d3f-43c5-a130-9757b588c6c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:24.346857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f458ce-9d3f-43c5-a130-9757b588c6c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:24.347128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f458ce-9d3f-43c5-a130-9757b588c6c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:24.356836 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f458ce-9d3f-43c5-a130-9757b588c6c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:24.358312 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f458ce-9d3f-43c5-a130-9757b588c6c0 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:24.369113 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1264/2540] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:24.649745 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563247b-e09b-4df7-b29f-c152b51918a9 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:24.651696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4563247b-e09b-4df7-b29f-c152b51918a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:24.665990 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563247b-e09b-4df7-b29f-c152b51918a9 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:24.671529 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fe1a5-80d3-4002-be6b-ba795d7c7845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:24.672012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fe1a5-80d3-4002-be6b-ba795d7c7845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:24.672270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fe1a5-80d3-4002-be6b-ba795d7c7845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:24.672739 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fe1a5-80d3-4002-be6b-ba795d7c7845 admin admin] PATH_INFO: `/v3/projects/b54810059a014d3d99879824cfcbb5f6` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:24.674367 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fe1a5-80d3-4002-be6b-ba795d7c7845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b54810059a014d3d99879824cfcbb5f6)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:24.676264 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fe1a5-80d3-4002-be6b-ba795d7c7845 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:24.723544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c22fe1a5-80d3-4002-be6b-ba795d7c7845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b54810059a014d3d99879824cfcbb5f6', 'request_id': 'req-c22fe1a5-80d3-4002-be6b-ba795d7c7845'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:24.731574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c22fe1a5-80d3-4002-be6b-ba795d7c7845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b54810059a014d3d99879824cfcbb5f6', 'request_id': 'req-c22fe1a5-80d3-4002-be6b-ba795d7c7845'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:24.737749 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c22fe1a5-80d3-4002-be6b-ba795d7c7845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b54810059a014d3d99879824cfcbb5f6', 'request_id': 'req-c22fe1a5-80d3-4002-be6b-ba795d7c7845'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:24.749495 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c22fe1a5-80d3-4002-be6b-ba795d7c7845 admin admin] The token cache is being invalidate because project b54810059a014d3d99879824cfcbb5f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:24.749784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c22fe1a5-80d3-4002-be6b-ba795d7c7845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:24.750425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c22fe1a5-80d3-4002-be6b-ba795d7c7845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:24.751947 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1277/2541] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:24 2022] DELETE /identity/v3/projects/b54810059a014d3d99879824cfcbb5f6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:24.756944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b318391-a8f7-4088-b5ae-028b3ea8339c None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:24.757238 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b318391-a8f7-4088-b5ae-028b3ea8339c None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:24.757502 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b318391-a8f7-4088-b5ae-028b3ea8339c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:24.777813 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-8b318391-a8f7-4088-b5ae-028b3ea8339c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:24.779047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-8b318391-a8f7-4088-b5ae-028b3ea8339c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:24.830351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:24.840739 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1265/2542] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:24.849162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205c7d4-e397-44e5-9bb3-86e0e559028e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:24.849389 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205c7d4-e397-44e5-9bb3-86e0e559028e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:24.849659 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205c7d4-e397-44e5-9bb3-86e0e559028e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:24.869832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b205c7d4-e397-44e5-9bb3-86e0e559028e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:24.871051 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b205c7d4-e397-44e5-9bb3-86e0e559028e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:24.879240 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1278/2543] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:24.883597 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b318391-a8f7-4088-b5ae-028b3ea8339c None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:24.893646 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b318391-a8f7-4088-b5ae-028b3ea8339c None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:24.898381 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55951be6-d145-405a-80ed-8c98a6ee58f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:24.899013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55951be6-d145-405a-80ed-8c98a6ee58f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:24.899287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55951be6-d145-405a-80ed-8c98a6ee58f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:24.899556 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55951be6-d145-405a-80ed-8c98a6ee58f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:24.900207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55951be6-d145-405a-80ed-8c98a6ee58f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:16:24.900506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55951be6-d145-405a-80ed-8c98a6ee58f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:24.902107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55951be6-d145-405a-80ed-8c98a6ee58f8 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:24.917827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1266/2544] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:16:24.919179 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205c7d4-e397-44e5-9bb3-86e0e559028e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:24.929508 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205c7d4-e397-44e5-9bb3-86e0e559028e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:24.934531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402caedf-43e5-4c7a-a079-c6a758e9d6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:24.935141 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402caedf-43e5-4c7a-a079-c6a758e9d6fe admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:24.935412 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402caedf-43e5-4c7a-a079-c6a758e9d6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:24.935679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402caedf-43e5-4c7a-a079-c6a758e9d6fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:24.936461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402caedf-43e5-4c7a-a079-c6a758e9d6fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:24.938439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402caedf-43e5-4c7a-a079-c6a758e9d6fe admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:24.967499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1279/2545] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:24 2022] POST /identity/v3/projects => generated 357 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:24.972502 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55951be6-d145-405a-80ed-8c98a6ee58f8 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:24.983113 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55951be6-d145-405a-80ed-8c98a6ee58f8 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:25.041412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e309793-a071-44c3-ad18-b7e1a5b7a86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:25.042023 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e309793-a071-44c3-ad18-b7e1a5b7a86d admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:25.042332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e309793-a071-44c3-ad18-b7e1a5b7a86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:25.042602 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e309793-a071-44c3-ad18-b7e1a5b7a86d admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:25.043228 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e309793-a071-44c3-ad18-b7e1a5b7a86d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:25.045136 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e309793-a071-44c3-ad18-b7e1a5b7a86d admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:25.068312 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1267/2546] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:24 2022] POST /identity/v3/users => generated 279 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:25.072700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402caedf-43e5-4c7a-a079-c6a758e9d6fe admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:25.083448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402caedf-43e5-4c7a-a079-c6a758e9d6fe admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:25.088060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74000c6-5ea3-414b-a69b-0766044bc735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:25.088654 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74000c6-5ea3-414b-a69b-0766044bc735 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:25.088933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74000c6-5ea3-414b-a69b-0766044bc735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:25.089205 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74000c6-5ea3-414b-a69b-0766044bc735 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:25.089848 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74000c6-5ea3-414b-a69b-0766044bc735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:25.091452 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74000c6-5ea3-414b-a69b-0766044bc735 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:25.102262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1280/2547] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:16:25.106213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e309793-a071-44c3-ad18-b7e1a5b7a86d admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:25.116679 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e309793-a071-44c3-ad18-b7e1a5b7a86d admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:25.121944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadb996-3f21-4506-8355-b6d6968ad1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:25.122575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadb996-3f21-4506-8355-b6d6968ad1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:25.122856 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadb996-3f21-4506-8355-b6d6968ad1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:25.123130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadb996-3f21-4506-8355-b6d6968ad1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:25.123777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffadb996-3f21-4506-8355-b6d6968ad1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:25.125454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffadb996-3f21-4506-8355-b6d6968ad1a2 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:25.136808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1268/2548] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:25.141655 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74000c6-5ea3-414b-a69b-0766044bc735 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:25.152403 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74000c6-5ea3-414b-a69b-0766044bc735 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:25.157056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83fcad-55fe-4405-b43a-d27a1dc540be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:25.157787 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83fcad-55fe-4405-b43a-d27a1dc540be admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:25.158911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83fcad-55fe-4405-b43a-d27a1dc540be admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:25.159193 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83fcad-55fe-4405-b43a-d27a1dc540be admin admin] PATH_INFO: `/v3/projects/a63f936a0e3f4d049a17bdc297de9918/users/9eac33793e544f2aa2b4dfcedbbaa800/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:25.174308 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83fcad-55fe-4405-b43a-d27a1dc540be admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63f936a0e3f4d049a17bdc297de9918, user_id=9eac33793e544f2aa2b4dfcedbbaa800, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:25.177014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83fcad-55fe-4405-b43a-d27a1dc540be admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:25.191808 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1281/2549] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:25 2022] PUT /identity/v3/projects/a63f936a0e3f4d049a17bdc297de9918/users/9eac33793e544f2aa2b4dfcedbbaa800/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:25.203865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1e0d1-1fac-4c9f-a081-72fca80461c0 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:25.204253 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1e0d1-1fac-4c9f-a081-72fca80461c0 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:25.204525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1e0d1-1fac-4c9f-a081-72fca80461c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:25.240327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-47d1e0d1-1fac-4c9f-a081-72fca80461c0 None None] MFA Rules not processed for user `9eac33793e544f2aa2b4dfcedbbaa800`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:25.241744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-47d1e0d1-1fac-4c9f-a081-72fca80461c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:25.324719 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1269/2550] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:25.329415 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83fcad-55fe-4405-b43a-d27a1dc540be admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:25.397097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83fcad-55fe-4405-b43a-d27a1dc540be admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:25.401830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6457d-3b1b-444d-922d-3281a776f913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:25.402460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6457d-3b1b-444d-922d-3281a776f913 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:25.402741 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6457d-3b1b-444d-922d-3281a776f913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:25.403010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6457d-3b1b-444d-922d-3281a776f913 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:25.403654 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6457d-3b1b-444d-922d-3281a776f913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:25.405599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6457d-3b1b-444d-922d-3281a776f913 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:25.432787 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1282/2551] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:25 2022] POST /identity/v3/projects => generated 355 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:25.436961 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1e0d1-1fac-4c9f-a081-72fca80461c0 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:25.447664 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1e0d1-1fac-4c9f-a081-72fca80461c0 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:25.496515 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:25.504354 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b22d05-06e8-48ee-9a7c-179193d64716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:25.504955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b22d05-06e8-48ee-9a7c-179193d64716 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:25.505232 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b22d05-06e8-48ee-9a7c-179193d64716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:25.505496 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b22d05-06e8-48ee-9a7c-179193d64716 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:25.506150 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b22d05-06e8-48ee-9a7c-179193d64716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:25.508093 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b22d05-06e8-48ee-9a7c-179193d64716 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:25.529668 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1270/2552] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:25 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:25.534296 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6457d-3b1b-444d-922d-3281a776f913 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:25.539649 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:25.546226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6457d-3b1b-444d-922d-3281a776f913 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:25.552677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b6798-95e4-4467-a2ca-ca3e1883a99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:25.553281 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b6798-95e4-4467-a2ca-ca3e1883a99a admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:25.553557 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b6798-95e4-4467-a2ca-ca3e1883a99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:25.553829 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b6798-95e4-4467-a2ca-ca3e1883a99a admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:25.554509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b6798-95e4-4467-a2ca-ca3e1883a99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:25.556094 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b6798-95e4-4467-a2ca-ca3e1883a99a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:25.566882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1283/2553] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:25.571066 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b22d05-06e8-48ee-9a7c-179193d64716 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:25.581021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b22d05-06e8-48ee-9a7c-179193d64716 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:25.585840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f509dc8-a397-4941-b93b-31ddf8ea20a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:25.586551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f509dc8-a397-4941-b93b-31ddf8ea20a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:25.586827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f509dc8-a397-4941-b93b-31ddf8ea20a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:25.587103 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f509dc8-a397-4941-b93b-31ddf8ea20a4 admin admin] PATH_INFO: `/v3/projects/82ea1a77d29d442195d0601ddceaa943/users/9ab986932e7c4f4b89f74084b1058add/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:25.602356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f509dc8-a397-4941-b93b-31ddf8ea20a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ea1a77d29d442195d0601ddceaa943, user_id=9ab986932e7c4f4b89f74084b1058add, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:25.605104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f509dc8-a397-4941-b93b-31ddf8ea20a4 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:25.619140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1271/2554] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:25 2022] PUT /identity/v3/projects/82ea1a77d29d442195d0601ddceaa943/users/9ab986932e7c4f4b89f74084b1058add/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:25.623143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b6798-95e4-4467-a2ca-ca3e1883a99a admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:25.633145 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b6798-95e4-4467-a2ca-ca3e1883a99a admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:25.689927 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481c3ca-4148-4bac-8aec-5f614c1e4e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:25.690561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481c3ca-4148-4bac-8aec-5f614c1e4e20 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:25.690841 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481c3ca-4148-4bac-8aec-5f614c1e4e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:25.691113 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481c3ca-4148-4bac-8aec-5f614c1e4e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:25.691753 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481c3ca-4148-4bac-8aec-5f614c1e4e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:25.693331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481c3ca-4148-4bac-8aec-5f614c1e4e20 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:25.702990 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1284/2555] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:25.707895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f509dc8-a397-4941-b93b-31ddf8ea20a4 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:25.718205 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f509dc8-a397-4941-b93b-31ddf8ea20a4 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:25.723414 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d4cdb-c159-446a-8d62-232ff88326c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:25.724011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d4cdb-c159-446a-8d62-232ff88326c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:25.724290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d4cdb-c159-446a-8d62-232ff88326c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:25.724607 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d4cdb-c159-446a-8d62-232ff88326c0 admin admin] PATH_INFO: `/v3/projects/82ea1a77d29d442195d0601ddceaa943/users/9ab986932e7c4f4b89f74084b1058add/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:25.726835 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d4cdb-c159-446a-8d62-232ff88326c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ea1a77d29d442195d0601ddceaa943, user_id=9ab986932e7c4f4b89f74084b1058add, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:25.729593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d4cdb-c159-446a-8d62-232ff88326c0 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:25.740142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1272/2556] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:25 2022] PUT /identity/v3/projects/82ea1a77d29d442195d0601ddceaa943/users/9ab986932e7c4f4b89f74084b1058add/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:25.750750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cc4c9-943b-42c1-9d71-4e15f606474d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:25.751027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cc4c9-943b-42c1-9d71-4e15f606474d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:25.751294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cc4c9-943b-42c1-9d71-4e15f606474d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:25.778989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9c0cc4c9-943b-42c1-9d71-4e15f606474d None None] MFA Rules not processed for user `9ab986932e7c4f4b89f74084b1058add`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:25.780259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9c0cc4c9-943b-42c1-9d71-4e15f606474d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:25.851609 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1285/2557] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:25.862863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d4cdb-c159-446a-8d62-232ff88326c0 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:25.864562 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a31d4cdb-c159-446a-8d62-232ff88326c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:25.936751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d4cdb-c159-446a-8d62-232ff88326c0 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:25.940778 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01627af7-18c7-4c81-bd76-6d3833090010 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:25.941401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01627af7-18c7-4c81-bd76-6d3833090010 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:25.941684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01627af7-18c7-4c81-bd76-6d3833090010 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:25.941984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01627af7-18c7-4c81-bd76-6d3833090010 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:25.951854 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01627af7-18c7-4c81-bd76-6d3833090010 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:25.953364 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01627af7-18c7-4c81-bd76-6d3833090010 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:25.964376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1273/2558] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:26.045940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cc4c9-943b-42c1-9d71-4e15f606474d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:26.055907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cc4c9-943b-42c1-9d71-4e15f606474d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:26.059556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc58334-9fad-40e6-9e5a-9abb6b623bed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:26.060375 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc58334-9fad-40e6-9e5a-9abb6b623bed service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:26.060836 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc58334-9fad-40e6-9e5a-9abb6b623bed service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:26.061285 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc58334-9fad-40e6-9e5a-9abb6b623bed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:26.121222 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc58334-9fad-40e6-9e5a-9abb6b623bed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:26.122609 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc58334-9fad-40e6-9e5a-9abb6b623bed service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:26.133120 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1286/2559] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:16:35.933935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01627af7-18c7-4c81-bd76-6d3833090010 service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:35.935601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-01627af7-18c7-4c81-bd76-6d3833090010 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:36.006322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01627af7-18c7-4c81-bd76-6d3833090010 service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:36.009943 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18486402-4fc3-4fce-9343-d8927f932f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:36.010586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18486402-4fc3-4fce-9343-d8927f932f3d service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:36.011023 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18486402-4fc3-4fce-9343-d8927f932f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:36.011300 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18486402-4fc3-4fce-9343-d8927f932f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:36.013063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-18486402-4fc3-4fce-9343-d8927f932f3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:36.077554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18486402-4fc3-4fce-9343-d8927f932f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:36.078954 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18486402-4fc3-4fce-9343-d8927f932f3d service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:36.084513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:36.090230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1274/2560] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:36.128576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:39.308874 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc58334-9fad-40e6-9e5a-9abb6b623bed service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:39.310728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ffc58334-9fad-40e6-9e5a-9abb6b623bed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:39.325637 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc58334-9fad-40e6-9e5a-9abb6b623bed service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:39.330753 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75c4d6-a97c-4645-bb50-220ee5740d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:39.331375 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75c4d6-a97c-4645-bb50-220ee5740d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:39.331652 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75c4d6-a97c-4645-bb50-220ee5740d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:39.331922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75c4d6-a97c-4645-bb50-220ee5740d74 admin admin] PATH_INFO: `/v3/users/16c99b5db9794dff83ea1b67fd238655` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:39.346781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75c4d6-a97c-4645-bb50-220ee5740d74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16c99b5db9794dff83ea1b67fd238655)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:39.348723 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75c4d6-a97c-4645-bb50-220ee5740d74 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:39.394574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5e75c4d6-a97c-4645-bb50-220ee5740d74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16c99b5db9794dff83ea1b67fd238655'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:39.395224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5e75c4d6-a97c-4645-bb50-220ee5740d74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16c99b5db9794dff83ea1b67fd238655'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:39.402317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5e75c4d6-a97c-4645-bb50-220ee5740d74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16c99b5db9794dff83ea1b67fd238655'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:39.410013 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5e75c4d6-a97c-4645-bb50-220ee5740d74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16c99b5db9794dff83ea1b67fd238655'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:39.421969 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5e75c4d6-a97c-4645-bb50-220ee5740d74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16c99b5db9794dff83ea1b67fd238655'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:39.427935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1287/2561] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:39 2022] DELETE /identity/v3/users/16c99b5db9794dff83ea1b67fd238655 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:39.604730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18486402-4fc3-4fce-9343-d8927f932f3d service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:39.615912 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18486402-4fc3-4fce-9343-d8927f932f3d service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:39.687315 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922046a-7942-471c-802a-f6ef94fd129a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:39.688082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922046a-7942-471c-802a-f6ef94fd129a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:39.688360 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922046a-7942-471c-802a-f6ef94fd129a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:39.688630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922046a-7942-471c-802a-f6ef94fd129a admin admin] PATH_INFO: `/v3/projects/51e38a10fcf8400394897df5f65428b0` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:39.700068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922046a-7942-471c-802a-f6ef94fd129a admin admin] RBAC: Authorizing `identity:delete_project(project_id=51e38a10fcf8400394897df5f65428b0)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:39.702013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922046a-7942-471c-802a-f6ef94fd129a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:39.762010 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d922046a-7942-471c-802a-f6ef94fd129a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51e38a10fcf8400394897df5f65428b0', 'request_id': 'req-d922046a-7942-471c-802a-f6ef94fd129a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:39.770540 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d922046a-7942-471c-802a-f6ef94fd129a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51e38a10fcf8400394897df5f65428b0', 'request_id': 'req-d922046a-7942-471c-802a-f6ef94fd129a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:39.776420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d922046a-7942-471c-802a-f6ef94fd129a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51e38a10fcf8400394897df5f65428b0', 'request_id': 'req-d922046a-7942-471c-802a-f6ef94fd129a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:39.788858 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d922046a-7942-471c-802a-f6ef94fd129a admin admin] The token cache is being invalidate because project 51e38a10fcf8400394897df5f65428b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:39.789145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d922046a-7942-471c-802a-f6ef94fd129a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:39.789793 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d922046a-7942-471c-802a-f6ef94fd129a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:39.791446 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1275/2562] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:39 2022] DELETE /identity/v3/projects/51e38a10fcf8400394897df5f65428b0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:39.796478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73861970-ea6a-41f3-80ab-e929f22715e2 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:39.796866 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73861970-ea6a-41f3-80ab-e929f22715e2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:39.797265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73861970-ea6a-41f3-80ab-e929f22715e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:39.817818 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-73861970-ea6a-41f3-80ab-e929f22715e2 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:39.819116 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-73861970-ea6a-41f3-80ab-e929f22715e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:39.887066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1288/2563] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:39.895830 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6e936-bcaa-48f8-b301-3a4804664c25 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:39.896133 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6e936-bcaa-48f8-b301-3a4804664c25 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:39.896432 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6e936-bcaa-48f8-b301-3a4804664c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:39.916845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-7cc6e936-bcaa-48f8-b301-3a4804664c25 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:39.918078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7cc6e936-bcaa-48f8-b301-3a4804664c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:39.926940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1276/2564] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:39.931289 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73861970-ea6a-41f3-80ab-e929f22715e2 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:39.941327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73861970-ea6a-41f3-80ab-e929f22715e2 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:39.946697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd2e26-9353-4fdf-8636-89f074ad99fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:39.947718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd2e26-9353-4fdf-8636-89f074ad99fd admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:39.948249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd2e26-9353-4fdf-8636-89f074ad99fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:39.948714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd2e26-9353-4fdf-8636-89f074ad99fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:39.949965 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd2e26-9353-4fdf-8636-89f074ad99fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:16:39.950567 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd2e26-9353-4fdf-8636-89f074ad99fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:39.952596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd2e26-9353-4fdf-8636-89f074ad99fd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:39.966785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1289/2565] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:16:39.970694 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6e936-bcaa-48f8-b301-3a4804664c25 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:39.979929 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6e936-bcaa-48f8-b301-3a4804664c25 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:39.984922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914ee56-cb4a-4b7e-b56f-989205c84b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:39.985516 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914ee56-cb4a-4b7e-b56f-989205c84b85 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:39.986321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914ee56-cb4a-4b7e-b56f-989205c84b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:39.986601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914ee56-cb4a-4b7e-b56f-989205c84b85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:39.987233 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914ee56-cb4a-4b7e-b56f-989205c84b85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:39.989134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914ee56-cb4a-4b7e-b56f-989205c84b85 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:40.017340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1277/2566] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:39 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:40.021643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd2e26-9353-4fdf-8636-89f074ad99fd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:40.031679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd2e26-9353-4fdf-8636-89f074ad99fd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:40.082247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:40.095458 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020414e-d81b-4987-9e1f-856e6c88f4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:40.096064 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020414e-d81b-4987-9e1f-856e6c88f4bb admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:40.096340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020414e-d81b-4987-9e1f-856e6c88f4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:40.096611 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020414e-d81b-4987-9e1f-856e6c88f4bb admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:40.097241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020414e-d81b-4987-9e1f-856e6c88f4bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:40.099249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020414e-d81b-4987-9e1f-856e6c88f4bb admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:40.120505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1290/2567] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:40 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:40.124808 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914ee56-cb4a-4b7e-b56f-989205c84b85 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:40.135766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914ee56-cb4a-4b7e-b56f-989205c84b85 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:40.140373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d8ab8-91d3-4b87-b17c-c03c91233fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:40.140965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d8ab8-91d3-4b87-b17c-c03c91233fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:40.141243 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d8ab8-91d3-4b87-b17c-c03c91233fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:40.141511 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d8ab8-91d3-4b87-b17c-c03c91233fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:40.142188 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d8ab8-91d3-4b87-b17c-c03c91233fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:40.144083 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d8ab8-91d3-4b87-b17c-c03c91233fd0 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:40.154251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1278/2568] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:40.158366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020414e-d81b-4987-9e1f-856e6c88f4bb admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:40.168881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020414e-d81b-4987-9e1f-856e6c88f4bb admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:40.175083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca76f3d-a362-4885-8264-08cb2c10808c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:40.175683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca76f3d-a362-4885-8264-08cb2c10808c admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:40.175961 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca76f3d-a362-4885-8264-08cb2c10808c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:40.176238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca76f3d-a362-4885-8264-08cb2c10808c admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:40.176878 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca76f3d-a362-4885-8264-08cb2c10808c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:40.178866 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca76f3d-a362-4885-8264-08cb2c10808c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:40.188719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1291/2569] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:16:40.193835 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d8ab8-91d3-4b87-b17c-c03c91233fd0 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:40.204600 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d8ab8-91d3-4b87-b17c-c03c91233fd0 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:40.209905 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f26e6f-4cc6-41c9-a604-79e4c37b06c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:40.210685 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f26e6f-4cc6-41c9-a604-79e4c37b06c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:40.211071 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f26e6f-4cc6-41c9-a604-79e4c37b06c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:40.211448 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f26e6f-4cc6-41c9-a604-79e4c37b06c3 admin admin] PATH_INFO: `/v3/projects/f71a2692f56c48c6928c269fa7f3b1dc/users/42fc6cd085e84905ad04c0b5a4450cfd/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:40.227097 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f26e6f-4cc6-41c9-a604-79e4c37b06c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71a2692f56c48c6928c269fa7f3b1dc, user_id=42fc6cd085e84905ad04c0b5a4450cfd, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:40.229919 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f26e6f-4cc6-41c9-a604-79e4c37b06c3 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:40.244468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1279/2570] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:40 2022] PUT /identity/v3/projects/f71a2692f56c48c6928c269fa7f3b1dc/users/42fc6cd085e84905ad04c0b5a4450cfd/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:40.255618 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246fd79-91b1-4813-88f8-4568503f91fe None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:40.255900 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246fd79-91b1-4813-88f8-4568503f91fe None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:40.256168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246fd79-91b1-4813-88f8-4568503f91fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:40.284446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b246fd79-91b1-4813-88f8-4568503f91fe None None] MFA Rules not processed for user `42fc6cd085e84905ad04c0b5a4450cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:40.285649 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b246fd79-91b1-4813-88f8-4568503f91fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:40.354291 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1292/2571] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:40 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:40.358973 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f26e6f-4cc6-41c9-a604-79e4c37b06c3 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:40.382021 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:40.431436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f26e6f-4cc6-41c9-a604-79e4c37b06c3 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:40.437252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66263e-e1d8-4214-87f1-d97152b2b782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:40.438332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66263e-e1d8-4214-87f1-d97152b2b782 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:40.438681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66263e-e1d8-4214-87f1-d97152b2b782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:40.439487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66263e-e1d8-4214-87f1-d97152b2b782 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:40.440191 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa66263e-e1d8-4214-87f1-d97152b2b782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:40.442513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa66263e-e1d8-4214-87f1-d97152b2b782 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:40.469730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1280/2572] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:40 2022] POST /identity/v3/projects => generated 405 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:40.473903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246fd79-91b1-4813-88f8-4568503f91fe None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:40.485001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246fd79-91b1-4813-88f8-4568503f91fe None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:40.543383 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4cd75-6621-4452-a44c-49cbb3a24430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:40.543995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4cd75-6621-4452-a44c-49cbb3a24430 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:40.544270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4cd75-6621-4452-a44c-49cbb3a24430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:40.544540 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4cd75-6621-4452-a44c-49cbb3a24430 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:40.545166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4cd75-6621-4452-a44c-49cbb3a24430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:40.547131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4cd75-6621-4452-a44c-49cbb3a24430 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:40.568934 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1293/2573] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:40 2022] POST /identity/v3/users => generated 309 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:40.573003 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66263e-e1d8-4214-87f1-d97152b2b782 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:40.583920 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66263e-e1d8-4214-87f1-d97152b2b782 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:40.588558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9230f3-1445-4e9d-9656-dcd744a21967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:40.589157 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9230f3-1445-4e9d-9656-dcd744a21967 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:40.589437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9230f3-1445-4e9d-9656-dcd744a21967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:40.589711 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9230f3-1445-4e9d-9656-dcd744a21967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:40.590418 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9230f3-1445-4e9d-9656-dcd744a21967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:40.592006 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9230f3-1445-4e9d-9656-dcd744a21967 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:40.603537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1281/2574] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:40.607720 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4cd75-6621-4452-a44c-49cbb3a24430 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:40.620001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4cd75-6621-4452-a44c-49cbb3a24430 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:40.624854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d344e-ac42-4526-bfe4-e8a95b1e3841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:40.625517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d344e-ac42-4526-bfe4-e8a95b1e3841 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:40.625790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d344e-ac42-4526-bfe4-e8a95b1e3841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:40.626059 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d344e-ac42-4526-bfe4-e8a95b1e3841 admin admin] PATH_INFO: `/v3/projects/eeb9363a4db6419cb655b5b09595c302/users/b69a4a884fb44d9a88f04cfbecb14adc/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:40.641423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d344e-ac42-4526-bfe4-e8a95b1e3841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeb9363a4db6419cb655b5b09595c302, user_id=b69a4a884fb44d9a88f04cfbecb14adc, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:40.644704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d344e-ac42-4526-bfe4-e8a95b1e3841 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:40.657722 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1294/2575] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:40 2022] PUT /identity/v3/projects/eeb9363a4db6419cb655b5b09595c302/users/b69a4a884fb44d9a88f04cfbecb14adc/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:40.661924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9230f3-1445-4e9d-9656-dcd744a21967 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:40.672277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9230f3-1445-4e9d-9656-dcd744a21967 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:40.727541 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b61cf-f5f9-46db-9fc9-f7df8230b5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:40.728148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b61cf-f5f9-46db-9fc9-f7df8230b5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:40.728608 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b61cf-f5f9-46db-9fc9-f7df8230b5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:40.728845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b61cf-f5f9-46db-9fc9-f7df8230b5c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:40.729412 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b61cf-f5f9-46db-9fc9-f7df8230b5c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:40.731054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b61cf-f5f9-46db-9fc9-f7df8230b5c5 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:40.742215 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1282/2576] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:40.746614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d344e-ac42-4526-bfe4-e8a95b1e3841 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:40.757625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d344e-ac42-4526-bfe4-e8a95b1e3841 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:40.762250 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa0c79-1dda-403e-b429-993b794716ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:40.762904 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa0c79-1dda-403e-b429-993b794716ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:40.763178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa0c79-1dda-403e-b429-993b794716ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:40.763450 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa0c79-1dda-403e-b429-993b794716ef admin admin] PATH_INFO: `/v3/projects/eeb9363a4db6419cb655b5b09595c302/users/b69a4a884fb44d9a88f04cfbecb14adc/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:40.765364 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa0c79-1dda-403e-b429-993b794716ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeb9363a4db6419cb655b5b09595c302, user_id=b69a4a884fb44d9a88f04cfbecb14adc, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:40.768016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa0c79-1dda-403e-b429-993b794716ef admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:40.778235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1295/2577] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:40 2022] PUT /identity/v3/projects/eeb9363a4db6419cb655b5b09595c302/users/b69a4a884fb44d9a88f04cfbecb14adc/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:40.789725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdab32-f49c-471e-a2f8-d497ede29977 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:40.790013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdab32-f49c-471e-a2f8-d497ede29977 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:40.790328 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdab32-f49c-471e-a2f8-d497ede29977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:40.819828 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d4fdab32-f49c-471e-a2f8-d497ede29977 None None] MFA Rules not processed for user `b69a4a884fb44d9a88f04cfbecb14adc`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:40.821096 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d4fdab32-f49c-471e-a2f8-d497ede29977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:40.891213 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1283/2578] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:40 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:40.902226 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa0c79-1dda-403e-b429-993b794716ef admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:40.903945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-93fa0c79-1dda-403e-b429-993b794716ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:40.914826 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:40.962255 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa0c79-1dda-403e-b429-993b794716ef admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:40.966643 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e732a3-7d15-4236-b3c4-035d8f4db65f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:40.967252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e732a3-7d15-4236-b3c4-035d8f4db65f service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:40.967531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e732a3-7d15-4236-b3c4-035d8f4db65f service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:40.967801 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e732a3-7d15-4236-b3c4-035d8f4db65f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:40.977271 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e732a3-7d15-4236-b3c4-035d8f4db65f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:40.978828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e732a3-7d15-4236-b3c4-035d8f4db65f service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:40.989487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1296/2579] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:40 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:41.148139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdab32-f49c-471e-a2f8-d497ede29977 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:41.163785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:41.223387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdab32-f49c-471e-a2f8-d497ede29977 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:41.230397 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32cb123-79ed-430c-a37d-372c7a67bced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:41.231048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cb123-79ed-430c-a37d-372c7a67bced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:41.231455 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cb123-79ed-430c-a37d-372c7a67bced admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:41.231745 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cb123-79ed-430c-a37d-372c7a67bced admin admin] PATH_INFO: `/v3/users/42fc6cd085e84905ad04c0b5a4450cfd` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:41.232906 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32cb123-79ed-430c-a37d-372c7a67bced admin admin] RBAC: Authorizing `identity:delete_user(user_id=42fc6cd085e84905ad04c0b5a4450cfd)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:41.238769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32cb123-79ed-430c-a37d-372c7a67bced admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:41.255082 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e732a3-7d15-4236-b3c4-035d8f4db65f service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:41.256759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-93e732a3-7d15-4236-b3c4-035d8f4db65f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:41.270741 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e732a3-7d15-4236-b3c4-035d8f4db65f service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:41.276416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a538d7-3776-42e0-bc20-5692e7c50b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:41.277121 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a538d7-3776-42e0-bc20-5692e7c50b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:41.277406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a538d7-3776-42e0-bc20-5692e7c50b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:41.277670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a538d7-3776-42e0-bc20-5692e7c50b70 admin admin] PATH_INFO: `/v3/users/9eac33793e544f2aa2b4dfcedbbaa800` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:41.278834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a538d7-3776-42e0-bc20-5692e7c50b70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eac33793e544f2aa2b4dfcedbbaa800)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:41.280992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a538d7-3776-42e0-bc20-5692e7c50b70 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:41.290526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f32cb123-79ed-430c-a37d-372c7a67bced admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42fc6cd085e84905ad04c0b5a4450cfd'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:41.300563 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f32cb123-79ed-430c-a37d-372c7a67bced admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42fc6cd085e84905ad04c0b5a4450cfd'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:41.314139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f32cb123-79ed-430c-a37d-372c7a67bced admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42fc6cd085e84905ad04c0b5a4450cfd'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:41.321177 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f32cb123-79ed-430c-a37d-372c7a67bced admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42fc6cd085e84905ad04c0b5a4450cfd'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:41.321870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f32cb123-79ed-430c-a37d-372c7a67bced admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42fc6cd085e84905ad04c0b5a4450cfd'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:41.327948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1284/2580] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:41 2022] DELETE /identity/v3/users/42fc6cd085e84905ad04c0b5a4450cfd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:41.329079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-92a538d7-3776-42e0-bc20-5692e7c50b70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eac33793e544f2aa2b4dfcedbbaa800'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:41.329753 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-92a538d7-3776-42e0-bc20-5692e7c50b70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eac33793e544f2aa2b4dfcedbbaa800'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:41.332038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9df924-b70b-4c90-9eee-3fafa8357c02 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:41.332324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9df924-b70b-4c90-9eee-3fafa8357c02 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:41.332598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9df924-b70b-4c90-9eee-3fafa8357c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:41.336912 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-92a538d7-3776-42e0-bc20-5692e7c50b70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eac33793e544f2aa2b4dfcedbbaa800'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:41.344975 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-92a538d7-3776-42e0-bc20-5692e7c50b70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eac33793e544f2aa2b4dfcedbbaa800'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:41.353779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-8b9df924-b70b-4c90-9eee-3fafa8357c02 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:41.355012 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-8b9df924-b70b-4c90-9eee-3fafa8357c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:41.357961 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-92a538d7-3776-42e0-bc20-5692e7c50b70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eac33793e544f2aa2b4dfcedbbaa800'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:41.366493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1297/2581] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:41 2022] DELETE /identity/v3/users/9eac33793e544f2aa2b4dfcedbbaa800 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:41.371310 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167619df-72cd-4e95-a689-604723131118 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:41.371593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167619df-72cd-4e95-a689-604723131118 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:41.371864 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167619df-72cd-4e95-a689-604723131118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:41.393392 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-167619df-72cd-4e95-a689-604723131118 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:41.394704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-167619df-72cd-4e95-a689-604723131118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:41.417720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1285/2582] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:41.429199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9df924-b70b-4c90-9eee-3fafa8357c02 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:41.430601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-8b9df924-b70b-4c90-9eee-3fafa8357c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:41.455390 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1298/2583] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:41.465125 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167619df-72cd-4e95-a689-604723131118 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:41.466293 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:41.466670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-167619df-72cd-4e95-a689-604723131118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:41.492820 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9df924-b70b-4c90-9eee-3fafa8357c02 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:41.496515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:41.528627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167619df-72cd-4e95-a689-604723131118 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:41.532382 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792435a1-b5fb-4d53-bc64-be2b51cd3437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:41.533188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792435a1-b5fb-4d53-bc64-be2b51cd3437 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:41.533463 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792435a1-b5fb-4d53-bc64-be2b51cd3437 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:41.533734 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792435a1-b5fb-4d53-bc64-be2b51cd3437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:41.543377 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792435a1-b5fb-4d53-bc64-be2b51cd3437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:41.544351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbe968-c6bd-41ac-9a6c-597f569084fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:41.544799 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792435a1-b5fb-4d53-bc64-be2b51cd3437 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:41.544968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbe968-c6bd-41ac-9a6c-597f569084fc service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:41.545244 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbe968-c6bd-41ac-9a6c-597f569084fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:41.545508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbe968-c6bd-41ac-9a6c-597f569084fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:41.555219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbe968-c6bd-41ac-9a6c-597f569084fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:41.555558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1299/2584] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:16:41.556669 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbe968-c6bd-41ac-9a6c-597f569084fc service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:41.567453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1286/2585] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:16:41.928330 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792435a1-b5fb-4d53-bc64-be2b51cd3437 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:41.930109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-792435a1-b5fb-4d53-bc64-be2b51cd3437 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:41.953994 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792435a1-b5fb-4d53-bc64-be2b51cd3437 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:41.965239 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285598e-8acb-4f56-8a00-f994d42c7ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:41.966136 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285598e-8acb-4f56-8a00-f994d42c7ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:41.966496 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285598e-8acb-4f56-8a00-f994d42c7ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:41.966774 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285598e-8acb-4f56-8a00-f994d42c7ec3 admin admin] PATH_INFO: `/v3/projects/a63f936a0e3f4d049a17bdc297de9918` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:41.967529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbe968-c6bd-41ac-9a6c-597f569084fc service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:41.968150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285598e-8acb-4f56-8a00-f994d42c7ec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63f936a0e3f4d049a17bdc297de9918)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:41.968998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2edbe968-c6bd-41ac-9a6c-597f569084fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:41.970316 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285598e-8acb-4f56-8a00-f994d42c7ec3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:41.991401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbe968-c6bd-41ac-9a6c-597f569084fc service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:41.998806 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27802282-8a24-4248-93c6-45c8fd811c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:41.999447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27802282-8a24-4248-93c6-45c8fd811c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:41.999730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27802282-8a24-4248-93c6-45c8fd811c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:42.000001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27802282-8a24-4248-93c6-45c8fd811c89 admin admin] PATH_INFO: `/v3/projects/f71a2692f56c48c6928c269fa7f3b1dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:42.001248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27802282-8a24-4248-93c6-45c8fd811c89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f71a2692f56c48c6928c269fa7f3b1dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:42.003790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27802282-8a24-4248-93c6-45c8fd811c89 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:42.024506 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b285598e-8acb-4f56-8a00-f994d42c7ec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63f936a0e3f4d049a17bdc297de9918', 'request_id': 'req-b285598e-8acb-4f56-8a00-f994d42c7ec3'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.032526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b285598e-8acb-4f56-8a00-f994d42c7ec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63f936a0e3f4d049a17bdc297de9918', 'request_id': 'req-b285598e-8acb-4f56-8a00-f994d42c7ec3'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.038592 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b285598e-8acb-4f56-8a00-f994d42c7ec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63f936a0e3f4d049a17bdc297de9918', 'request_id': 'req-b285598e-8acb-4f56-8a00-f994d42c7ec3'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.049844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b285598e-8acb-4f56-8a00-f994d42c7ec3 admin admin] The token cache is being invalidate because project a63f936a0e3f4d049a17bdc297de9918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:42.050151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b285598e-8acb-4f56-8a00-f994d42c7ec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.050797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b285598e-8acb-4f56-8a00-f994d42c7ec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.052387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1300/2586] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:41 2022] DELETE /identity/v3/projects/a63f936a0e3f4d049a17bdc297de9918 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:42.053073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-27802282-8a24-4248-93c6-45c8fd811c89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f71a2692f56c48c6928c269fa7f3b1dc', 'request_id': 'req-27802282-8a24-4248-93c6-45c8fd811c89'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.056698 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285598e-8acb-4f56-8a00-f994d42c7ec3 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:42.058166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b285598e-8acb-4f56-8a00-f994d42c7ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:42.060957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-27802282-8a24-4248-93c6-45c8fd811c89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f71a2692f56c48c6928c269fa7f3b1dc', 'request_id': 'req-27802282-8a24-4248-93c6-45c8fd811c89'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.067598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-27802282-8a24-4248-93c6-45c8fd811c89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f71a2692f56c48c6928c269fa7f3b1dc', 'request_id': 'req-27802282-8a24-4248-93c6-45c8fd811c89'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.078667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-27802282-8a24-4248-93c6-45c8fd811c89 admin admin] The token cache is being invalidate because project f71a2692f56c48c6928c269fa7f3b1dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:42.078950 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-27802282-8a24-4248-93c6-45c8fd811c89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.079680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-27802282-8a24-4248-93c6-45c8fd811c89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.081859 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1287/2587] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:41 2022] DELETE /identity/v3/projects/f71a2692f56c48c6928c269fa7f3b1dc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:42.085450 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27802282-8a24-4248-93c6-45c8fd811c89 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:42.086763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-27802282-8a24-4248-93c6-45c8fd811c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:42.125280 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285598e-8acb-4f56-8a00-f994d42c7ec3 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:42.155277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27802282-8a24-4248-93c6-45c8fd811c89 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:42.160222 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc042a5-3e57-4f47-bbf2-f6ff04045029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:42.160862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc042a5-3e57-4f47-bbf2-f6ff04045029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:42.161142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc042a5-3e57-4f47-bbf2-f6ff04045029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:42.161416 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc042a5-3e57-4f47-bbf2-f6ff04045029 admin admin] PATH_INFO: `/v3/users/b69a4a884fb44d9a88f04cfbecb14adc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:42.162659 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc042a5-3e57-4f47-bbf2-f6ff04045029 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b69a4a884fb44d9a88f04cfbecb14adc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:42.164754 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc042a5-3e57-4f47-bbf2-f6ff04045029 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:42.183338 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:42.183576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:42.191368 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701e614-64e2-4ac9-b9ec-aec937505ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:42.193272 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701e614-64e2-4ac9-b9ec-aec937505ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:42.193577 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701e614-64e2-4ac9-b9ec-aec937505ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:42.194251 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701e614-64e2-4ac9-b9ec-aec937505ae3 admin admin] PATH_INFO: `/v3/users/9ab986932e7c4f4b89f74084b1058add` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:42.195711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701e614-64e2-4ac9-b9ec-aec937505ae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab986932e7c4f4b89f74084b1058add)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:42.197830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701e614-64e2-4ac9-b9ec-aec937505ae3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:42.225176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2fc042a5-3e57-4f47-bbf2-f6ff04045029 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b69a4a884fb44d9a88f04cfbecb14adc'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.234080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2fc042a5-3e57-4f47-bbf2-f6ff04045029 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b69a4a884fb44d9a88f04cfbecb14adc'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.248276 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2fc042a5-3e57-4f47-bbf2-f6ff04045029 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b69a4a884fb44d9a88f04cfbecb14adc'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.251156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0701e614-64e2-4ac9-b9ec-aec937505ae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab986932e7c4f4b89f74084b1058add'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.252732 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0701e614-64e2-4ac9-b9ec-aec937505ae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab986932e7c4f4b89f74084b1058add'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.256530 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2fc042a5-3e57-4f47-bbf2-f6ff04045029 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b69a4a884fb44d9a88f04cfbecb14adc'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.257301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2fc042a5-3e57-4f47-bbf2-f6ff04045029 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b69a4a884fb44d9a88f04cfbecb14adc'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.260511 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0701e614-64e2-4ac9-b9ec-aec937505ae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab986932e7c4f4b89f74084b1058add'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.264186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1288/2588] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:42 2022] DELETE /identity/v3/users/b69a4a884fb44d9a88f04cfbecb14adc => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:42.273102 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0701e614-64e2-4ac9-b9ec-aec937505ae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab986932e7c4f4b89f74084b1058add'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.285946 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0701e614-64e2-4ac9-b9ec-aec937505ae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab986932e7c4f4b89f74084b1058add'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.292215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1301/2589] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:42 2022] DELETE /identity/v3/users/9ab986932e7c4f4b89f74084b1058add => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:42.668937 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc042a5-3e57-4f47-bbf2-f6ff04045029 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:42.670482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2fc042a5-3e57-4f47-bbf2-f6ff04045029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:42.728182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701e614-64e2-4ac9-b9ec-aec937505ae3 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:42.730900 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0701e614-64e2-4ac9-b9ec-aec937505ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:42.747240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc042a5-3e57-4f47-bbf2-f6ff04045029 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:42.803865 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701e614-64e2-4ac9-b9ec-aec937505ae3 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:42.808276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b0570-703f-4892-ac33-3124efd05dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:42.808416 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdf67e-526b-4e52-8716-c08bdbadd863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:42.808937 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b0570-703f-4892-ac33-3124efd05dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:42.809067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdf67e-526b-4e52-8716-c08bdbadd863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:42.809188 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b0570-703f-4892-ac33-3124efd05dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:42.809286 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdf67e-526b-4e52-8716-c08bdbadd863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:42.809434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b0570-703f-4892-ac33-3124efd05dfd admin admin] PATH_INFO: `/v3/projects/82ea1a77d29d442195d0601ddceaa943` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:42.809559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdf67e-526b-4e52-8716-c08bdbadd863 admin admin] PATH_INFO: `/v3/projects/eeb9363a4db6419cb655b5b09595c302` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:42.810572 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b0570-703f-4892-ac33-3124efd05dfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ea1a77d29d442195d0601ddceaa943)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:42.810690 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdf67e-526b-4e52-8716-c08bdbadd863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeb9363a4db6419cb655b5b09595c302)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:42.812465 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b0570-703f-4892-ac33-3124efd05dfd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:42.812601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdf67e-526b-4e52-8716-c08bdbadd863 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:42.859671 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:42.864414 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-50cdf67e-526b-4e52-8716-c08bdbadd863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeb9363a4db6419cb655b5b09595c302', 'request_id': 'req-50cdf67e-526b-4e52-8716-c08bdbadd863'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.869360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0d7b0570-703f-4892-ac33-3124efd05dfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ea1a77d29d442195d0601ddceaa943', 'request_id': 'req-0d7b0570-703f-4892-ac33-3124efd05dfd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.876141 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-50cdf67e-526b-4e52-8716-c08bdbadd863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeb9363a4db6419cb655b5b09595c302', 'request_id': 'req-50cdf67e-526b-4e52-8716-c08bdbadd863'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.880880 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0d7b0570-703f-4892-ac33-3124efd05dfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ea1a77d29d442195d0601ddceaa943', 'request_id': 'req-0d7b0570-703f-4892-ac33-3124efd05dfd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.887628 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-50cdf67e-526b-4e52-8716-c08bdbadd863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeb9363a4db6419cb655b5b09595c302', 'request_id': 'req-50cdf67e-526b-4e52-8716-c08bdbadd863'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.888355 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0d7b0570-703f-4892-ac33-3124efd05dfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ea1a77d29d442195d0601ddceaa943', 'request_id': 'req-0d7b0570-703f-4892-ac33-3124efd05dfd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.901499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0d7b0570-703f-4892-ac33-3124efd05dfd admin admin] The token cache is being invalidate because project 82ea1a77d29d442195d0601ddceaa943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:42.901855 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0d7b0570-703f-4892-ac33-3124efd05dfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.902430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-50cdf67e-526b-4e52-8716-c08bdbadd863 admin admin] The token cache is being invalidate because project eeb9363a4db6419cb655b5b09595c302 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:42.902575 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0d7b0570-703f-4892-ac33-3124efd05dfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.902847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-50cdf67e-526b-4e52-8716-c08bdbadd863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.903638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-50cdf67e-526b-4e52-8716-c08bdbadd863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:42.903938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1302/2590] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:42 2022] DELETE /identity/v3/projects/82ea1a77d29d442195d0601ddceaa943 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:42.908883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1289/2591] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:42 2022] DELETE /identity/v3/projects/eeb9363a4db6419cb655b5b09595c302 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:42.909093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c125ce7-32bb-4b03-98df-94c097c9346e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:42.909216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c125ce7-32bb-4b03-98df-94c097c9346e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:42.909448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c125ce7-32bb-4b03-98df-94c097c9346e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:42.913274 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759070d3-b47a-4f94-b643-aec20623cca2 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:42.913623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759070d3-b47a-4f94-b643-aec20623cca2 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:42.913961 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759070d3-b47a-4f94-b643-aec20623cca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:42.924068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:42.930009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-0c125ce7-32bb-4b03-98df-94c097c9346e None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:42.931908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0c125ce7-32bb-4b03-98df-94c097c9346e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:42.933959 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-759070d3-b47a-4f94-b643-aec20623cca2 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:42.935159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-759070d3-b47a-4f94-b643-aec20623cca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:42.993424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1303/2592] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:42.999382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1290/2593] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:43.003234 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcc878-7f98-4a9f-a94a-04b4ec159129 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.003518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcc878-7f98-4a9f-a94a-04b4ec159129 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.003789 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcc878-7f98-4a9f-a94a-04b4ec159129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.007586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f20df9-d0d3-46c7-abb2-05f010d615d5 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.007868 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f20df9-d0d3-46c7-abb2-05f010d615d5 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.008141 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f20df9-d0d3-46c7-abb2-05f010d615d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.024212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-33dcc878-7f98-4a9f-a94a-04b4ec159129 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:43.025421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-33dcc878-7f98-4a9f-a94a-04b4ec159129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:43.027180 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-93f20df9-d0d3-46c7-abb2-05f010d615d5 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:43.028336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-93f20df9-d0d3-46c7-abb2-05f010d615d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:43.033759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1304/2594] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:43.036648 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1291/2595] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:43.037978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcc878-7f98-4a9f-a94a-04b4ec159129 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.040735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f20df9-d0d3-46c7-abb2-05f010d615d5 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.048776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcc878-7f98-4a9f-a94a-04b4ec159129 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.051837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f20df9-d0d3-46c7-abb2-05f010d615d5 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.053803 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27157407-a99f-41a3-9f56-b9f84e661463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.055083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27157407-a99f-41a3-9f56-b9f84e661463 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.055083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27157407-a99f-41a3-9f56-b9f84e661463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.055083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27157407-a99f-41a3-9f56-b9f84e661463 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.055641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27157407-a99f-41a3-9f56-b9f84e661463 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:16:43.056037 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27157407-a99f-41a3-9f56-b9f84e661463 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.059170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba56a6-1d8e-4512-8f9d-393b7fd439f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.059170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba56a6-1d8e-4512-8f9d-393b7fd439f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.059170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba56a6-1d8e-4512-8f9d-393b7fd439f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.059170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba56a6-1d8e-4512-8f9d-393b7fd439f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.059817 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27157407-a99f-41a3-9f56-b9f84e661463 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.059991 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba56a6-1d8e-4512-8f9d-393b7fd439f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:16:43.059991 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba56a6-1d8e-4512-8f9d-393b7fd439f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.059991 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba56a6-1d8e-4512-8f9d-393b7fd439f9 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.072428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1292/2596] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:16:43.072678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1305/2597] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:16:43.077116 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba56a6-1d8e-4512-8f9d-393b7fd439f9 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.077532 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27157407-a99f-41a3-9f56-b9f84e661463 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.088674 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27157407-a99f-41a3-9f56-b9f84e661463 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.088852 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba56a6-1d8e-4512-8f9d-393b7fd439f9 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.092083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25accd2-ae4a-42f1-ac24-4380d980694f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.092512 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80f154-808a-4d6d-82d8-3332519e5113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.092679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25accd2-ae4a-42f1-ac24-4380d980694f admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.092960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25accd2-ae4a-42f1-ac24-4380d980694f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.093063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80f154-808a-4d6d-82d8-3332519e5113 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.093353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25accd2-ae4a-42f1-ac24-4380d980694f admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.093481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80f154-808a-4d6d-82d8-3332519e5113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.093640 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80f154-808a-4d6d-82d8-3332519e5113 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.093896 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25accd2-ae4a-42f1-ac24-4380d980694f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.094330 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80f154-808a-4d6d-82d8-3332519e5113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.095840 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25accd2-ae4a-42f1-ac24-4380d980694f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.096330 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80f154-808a-4d6d-82d8-3332519e5113 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.124846 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1293/2598] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:43 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:43.126266 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1306/2599] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:43 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:43.129480 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80f154-808a-4d6d-82d8-3332519e5113 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.129774 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25accd2-ae4a-42f1-ac24-4380d980694f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.194511 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80f154-808a-4d6d-82d8-3332519e5113 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.199478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25accd2-ae4a-42f1-ac24-4380d980694f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.199820 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a4221-0a4a-4da2-a0c4-48e691735386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.200459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a4221-0a4a-4da2-a0c4-48e691735386 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.200740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a4221-0a4a-4da2-a0c4-48e691735386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.201015 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a4221-0a4a-4da2-a0c4-48e691735386 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.201662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a4221-0a4a-4da2-a0c4-48e691735386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.203695 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a4221-0a4a-4da2-a0c4-48e691735386 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.204468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f8629-91b0-4392-95cd-ba4ef68143a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.205128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f8629-91b0-4392-95cd-ba4ef68143a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.205406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f8629-91b0-4392-95cd-ba4ef68143a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.205678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f8629-91b0-4392-95cd-ba4ef68143a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.206346 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f8629-91b0-4392-95cd-ba4ef68143a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.208243 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f8629-91b0-4392-95cd-ba4ef68143a4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.228955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1294/2600] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:43 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:43.230170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a4221-0a4a-4da2-a0c4-48e691735386 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.236371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1307/2601] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:43 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:43.240106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f8629-91b0-4392-95cd-ba4ef68143a4 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.240823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a4221-0a4a-4da2-a0c4-48e691735386 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.245665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69844f97-6ad9-4628-b5db-c2e8df87088b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.246286 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69844f97-6ad9-4628-b5db-c2e8df87088b admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.246559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69844f97-6ad9-4628-b5db-c2e8df87088b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.246826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69844f97-6ad9-4628-b5db-c2e8df87088b admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.247464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69844f97-6ad9-4628-b5db-c2e8df87088b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.249055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69844f97-6ad9-4628-b5db-c2e8df87088b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.250636 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f8629-91b0-4392-95cd-ba4ef68143a4 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.255923 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f7c3b-0cb1-4496-878d-f04087f4f83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.256515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f7c3b-0cb1-4496-878d-f04087f4f83f admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.256791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f7c3b-0cb1-4496-878d-f04087f4f83f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.257068 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f7c3b-0cb1-4496-878d-f04087f4f83f admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.257709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f7c3b-0cb1-4496-878d-f04087f4f83f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.259315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f7c3b-0cb1-4496-878d-f04087f4f83f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.260056 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1295/2602] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:16:43.263730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69844f97-6ad9-4628-b5db-c2e8df87088b admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.276214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1308/2603] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:16:43.276407 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69844f97-6ad9-4628-b5db-c2e8df87088b admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.277254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f7c3b-0cb1-4496-878d-f04087f4f83f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.278299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27760de7-410c-4772-9f33-a0f4f0341580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.278887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27760de7-410c-4772-9f33-a0f4f0341580 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.279160 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27760de7-410c-4772-9f33-a0f4f0341580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.279424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27760de7-410c-4772-9f33-a0f4f0341580 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.280055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27760de7-410c-4772-9f33-a0f4f0341580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.281622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27760de7-410c-4772-9f33-a0f4f0341580 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.283863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:43.288084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f7c3b-0cb1-4496-878d-f04087f4f83f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.293684 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a6a4a-3d01-4de2-accc-7fc4183785fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.294274 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1296/2604] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:16:43.297702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a6a4a-3d01-4de2-accc-7fc4183785fb admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.298017 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a6a4a-3d01-4de2-accc-7fc4183785fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.298837 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a6a4a-3d01-4de2-accc-7fc4183785fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.299235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27760de7-410c-4772-9f33-a0f4f0341580 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.299502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a6a4a-3d01-4de2-accc-7fc4183785fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.301095 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a6a4a-3d01-4de2-accc-7fc4183785fb admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.309004 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27760de7-410c-4772-9f33-a0f4f0341580 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.312119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1309/2605] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:16:43.313561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116310d-b18c-4ecf-817b-c63a4fed2d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.314799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116310d-b18c-4ecf-817b-c63a4fed2d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.315068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116310d-b18c-4ecf-817b-c63a4fed2d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.315333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116310d-b18c-4ecf-817b-c63a4fed2d29 admin admin] PATH_INFO: `/v3/projects/6c51a839a6994b19b470af006a05f8d7/users/a0e11d99fe8a49f7ab6afe58fb07dd6e/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.316910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a6a4a-3d01-4de2-accc-7fc4183785fb admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.326811 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a6a4a-3d01-4de2-accc-7fc4183785fb admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.330477 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116310d-b18c-4ecf-817b-c63a4fed2d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c51a839a6994b19b470af006a05f8d7, user_id=a0e11d99fe8a49f7ab6afe58fb07dd6e, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.332131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7f15e-ad1e-43d2-8e0b-a9d8e24f213c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.332777 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7f15e-ad1e-43d2-8e0b-a9d8e24f213c admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.333062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7f15e-ad1e-43d2-8e0b-a9d8e24f213c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.333174 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116310d-b18c-4ecf-817b-c63a4fed2d29 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.333462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7f15e-ad1e-43d2-8e0b-a9d8e24f213c admin admin] PATH_INFO: `/v3/projects/9ea9e1ab8e1741399df95f93f2ba2a73/users/9c7c1ce6f5044db4acde4159404bf4d6/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.347306 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1297/2606] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:43 2022] PUT /identity/v3/projects/6c51a839a6994b19b470af006a05f8d7/users/a0e11d99fe8a49f7ab6afe58fb07dd6e/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:43.358521 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1992aa-0482-4d37-a9de-d4a91fcb7c42 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.358814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1992aa-0482-4d37-a9de-d4a91fcb7c42 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.359086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1992aa-0482-4d37-a9de-d4a91fcb7c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.359500 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7f15e-ad1e-43d2-8e0b-a9d8e24f213c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea9e1ab8e1741399df95f93f2ba2a73, user_id=9c7c1ce6f5044db4acde4159404bf4d6, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.362390 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7f15e-ad1e-43d2-8e0b-a9d8e24f213c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.376388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1310/2607] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:43 2022] PUT /identity/v3/projects/9ea9e1ab8e1741399df95f93f2ba2a73/users/9c7c1ce6f5044db4acde4159404bf4d6/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:43.388210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe31a98-8ab4-4085-be41-17456b1755b6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.388489 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe31a98-8ab4-4085-be41-17456b1755b6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.388758 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe31a98-8ab4-4085-be41-17456b1755b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.394741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-8c1992aa-0482-4d37-a9de-d4a91fcb7c42 None None] MFA Rules not processed for user `a0e11d99fe8a49f7ab6afe58fb07dd6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:43.395959 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-8c1992aa-0482-4d37-a9de-d4a91fcb7c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:43.408615 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:43.419003 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-6fe31a98-8ab4-4085-be41-17456b1755b6 None None] MFA Rules not processed for user `9c7c1ce6f5044db4acde4159404bf4d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:43.420855 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6fe31a98-8ab4-4085-be41-17456b1755b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:43.462426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1298/2608] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:43.467197 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1992aa-0482-4d37-a9de-d4a91fcb7c42 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.489009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1311/2609] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:43.493453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe31a98-8ab4-4085-be41-17456b1755b6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.532016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1992aa-0482-4d37-a9de-d4a91fcb7c42 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.536966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4153fb3-fa4b-4aa7-89b7-f42682cd0169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.537610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4153fb3-fa4b-4aa7-89b7-f42682cd0169 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.537891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4153fb3-fa4b-4aa7-89b7-f42682cd0169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.538183 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4153fb3-fa4b-4aa7-89b7-f42682cd0169 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.538841 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4153fb3-fa4b-4aa7-89b7-f42682cd0169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.540985 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4153fb3-fa4b-4aa7-89b7-f42682cd0169 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.561518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe31a98-8ab4-4085-be41-17456b1755b6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.567268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba8bb4-f8be-4c48-aa15-e4743196b73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.567888 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba8bb4-f8be-4c48-aa15-e4743196b73a admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.568193 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba8bb4-f8be-4c48-aa15-e4743196b73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.568464 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba8bb4-f8be-4c48-aa15-e4743196b73a admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.569126 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba8bb4-f8be-4c48-aa15-e4743196b73a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.570420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1299/2610] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:43 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:43.571077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba8bb4-f8be-4c48-aa15-e4743196b73a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.574847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4153fb3-fa4b-4aa7-89b7-f42682cd0169 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.600061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1312/2611] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:43 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:43.604667 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba8bb4-f8be-4c48-aa15-e4743196b73a admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.628728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:43.652155 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4153fb3-fa4b-4aa7-89b7-f42682cd0169 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.672842 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba8bb4-f8be-4c48-aa15-e4743196b73a admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.677882 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df2663-cfa2-4a36-9a9d-7bdd3298c3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.678519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df2663-cfa2-4a36-9a9d-7bdd3298c3d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.678820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df2663-cfa2-4a36-9a9d-7bdd3298c3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.679207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df2663-cfa2-4a36-9a9d-7bdd3298c3d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.679860 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df2663-cfa2-4a36-9a9d-7bdd3298c3d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.681842 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df2663-cfa2-4a36-9a9d-7bdd3298c3d3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.706943 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1313/2612] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:43 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:43.711427 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df2663-cfa2-4a36-9a9d-7bdd3298c3d3 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.721896 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df2663-cfa2-4a36-9a9d-7bdd3298c3d3 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.727181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb992d-5fda-4709-898a-96d0f6173d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.727181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb992d-5fda-4709-898a-96d0f6173d4a admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.727181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb992d-5fda-4709-898a-96d0f6173d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.727181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb992d-5fda-4709-898a-96d0f6173d4a admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.728052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb992d-5fda-4709-898a-96d0f6173d4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.728144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fb38d-c69f-4a79-b8b5-8c9a40d4909f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.728144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fb38d-c69f-4a79-b8b5-8c9a40d4909f admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.728776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb992d-5fda-4709-898a-96d0f6173d4a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.728870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fb38d-c69f-4a79-b8b5-8c9a40d4909f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.728870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fb38d-c69f-4a79-b8b5-8c9a40d4909f admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.729177 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fb38d-c69f-4a79-b8b5-8c9a40d4909f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.730964 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fb38d-c69f-4a79-b8b5-8c9a40d4909f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.743195 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1314/2613] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:43.747804 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fb38d-c69f-4a79-b8b5-8c9a40d4909f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.753735 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:43.754972 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1300/2614] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:43 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:43.758705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb992d-5fda-4709-898a-96d0f6173d4a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.760975 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fb38d-c69f-4a79-b8b5-8c9a40d4909f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.766879 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c807399-7d20-4390-940d-44d94c2beee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.767553 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c807399-7d20-4390-940d-44d94c2beee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.767829 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c807399-7d20-4390-940d-44d94c2beee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.768104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c807399-7d20-4390-940d-44d94c2beee7 admin admin] PATH_INFO: `/v3/projects/299b3e4ec03947f7ae792f249eb166b5/users/6a53eb2d29b74e019acb8e59ef23e8d5/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.769562 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb992d-5fda-4709-898a-96d0f6173d4a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.774357 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af08f4-8af9-42be-b4bc-d63b26365a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.774961 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af08f4-8af9-42be-b4bc-d63b26365a64 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.775240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af08f4-8af9-42be-b4bc-d63b26365a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.775509 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af08f4-8af9-42be-b4bc-d63b26365a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.776154 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af08f4-8af9-42be-b4bc-d63b26365a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.777750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af08f4-8af9-42be-b4bc-d63b26365a64 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.784307 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c807399-7d20-4390-940d-44d94c2beee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=299b3e4ec03947f7ae792f249eb166b5, user_id=6a53eb2d29b74e019acb8e59ef23e8d5, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.787107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c807399-7d20-4390-940d-44d94c2beee7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.788063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1301/2615] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:16:43.792130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af08f4-8af9-42be-b4bc-d63b26365a64 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.802102 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1315/2616] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:43 2022] PUT /identity/v3/projects/299b3e4ec03947f7ae792f249eb166b5/users/6a53eb2d29b74e019acb8e59ef23e8d5/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:43.802590 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af08f4-8af9-42be-b4bc-d63b26365a64 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.805962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c807399-7d20-4390-940d-44d94c2beee7 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.858920 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992d7c3-0116-4e34-9bdf-0abde9c528d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.859672 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992d7c3-0116-4e34-9bdf-0abde9c528d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.859951 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992d7c3-0116-4e34-9bdf-0abde9c528d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.860221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992d7c3-0116-4e34-9bdf-0abde9c528d7 admin admin] PATH_INFO: `/v3/projects/63e2df0bf70d4d78b25b720067ab3b49/users/e302188b8b0f4ea596f5020666d9be1a/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.867990 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c807399-7d20-4390-940d-44d94c2beee7 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.873431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257d675-3739-41d6-9306-417e11b8c31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.874074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257d675-3739-41d6-9306-417e11b8c31e admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.874381 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257d675-3739-41d6-9306-417e11b8c31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.874655 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257d675-3739-41d6-9306-417e11b8c31e admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.875303 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257d675-3739-41d6-9306-417e11b8c31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.876391 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992d7c3-0116-4e34-9bdf-0abde9c528d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e2df0bf70d4d78b25b720067ab3b49, user_id=e302188b8b0f4ea596f5020666d9be1a, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.877512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257d675-3739-41d6-9306-417e11b8c31e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.879863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992d7c3-0116-4e34-9bdf-0abde9c528d7 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.888825 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1316/2617] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:43.893262 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257d675-3739-41d6-9306-417e11b8c31e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.895108 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1302/2618] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:43 2022] PUT /identity/v3/projects/63e2df0bf70d4d78b25b720067ab3b49/users/e302188b8b0f4ea596f5020666d9be1a/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:43.898566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992d7c3-0116-4e34-9bdf-0abde9c528d7 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.903478 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257d675-3739-41d6-9306-417e11b8c31e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.957447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:43.961948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992d7c3-0116-4e34-9bdf-0abde9c528d7 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:43.965307 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6665b2-2106-40fc-bc2a-fe4e08fca234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.966029 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6665b2-2106-40fc-bc2a-fe4e08fca234 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.966371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6665b2-2106-40fc-bc2a-fe4e08fca234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.966674 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6665b2-2106-40fc-bc2a-fe4e08fca234 admin admin] PATH_INFO: `/v3/projects/299b3e4ec03947f7ae792f249eb166b5/users/6a53eb2d29b74e019acb8e59ef23e8d5/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.968199 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e513ca-f699-4fdc-ae66-e1637cbb9eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:43.968635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6665b2-2106-40fc-bc2a-fe4e08fca234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=299b3e4ec03947f7ae792f249eb166b5, user_id=6a53eb2d29b74e019acb8e59ef23e8d5, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.969081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e513ca-f699-4fdc-ae66-e1637cbb9eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.969537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e513ca-f699-4fdc-ae66-e1637cbb9eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.969918 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e513ca-f699-4fdc-ae66-e1637cbb9eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:43.970724 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e513ca-f699-4fdc-ae66-e1637cbb9eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:43.971368 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6665b2-2106-40fc-bc2a-fe4e08fca234 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.972525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e513ca-f699-4fdc-ae66-e1637cbb9eb5 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:43.982759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1317/2619] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:43 2022] PUT /identity/v3/projects/299b3e4ec03947f7ae792f249eb166b5/users/6a53eb2d29b74e019acb8e59ef23e8d5/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:43.987353 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1303/2620] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:16:43.991169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6665b2-2106-40fc-bc2a-fe4e08fca234 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:43.993748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18739828-1f2e-4ad1-a405-ea66a7ec26f1 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:43.994040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18739828-1f2e-4ad1-a405-ea66a7ec26f1 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:43.994334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18739828-1f2e-4ad1-a405-ea66a7ec26f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:44.024092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-18739828-1f2e-4ad1-a405-ea66a7ec26f1 None None] MFA Rules not processed for user `6a53eb2d29b74e019acb8e59ef23e8d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:44.025382 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-18739828-1f2e-4ad1-a405-ea66a7ec26f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:44.062718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6665b2-2106-40fc-bc2a-fe4e08fca234 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:44.068497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bee63-5ad9-4544-845b-aa47f7ba623b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:44.069150 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bee63-5ad9-4544-845b-aa47f7ba623b admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:44.069424 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bee63-5ad9-4544-845b-aa47f7ba623b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:44.069693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bee63-5ad9-4544-845b-aa47f7ba623b admin admin] PATH_INFO: `/v3/projects/63e2df0bf70d4d78b25b720067ab3b49/users/e302188b8b0f4ea596f5020666d9be1a/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:44.071785 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bee63-5ad9-4544-845b-aa47f7ba623b admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e2df0bf70d4d78b25b720067ab3b49, user_id=e302188b8b0f4ea596f5020666d9be1a, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:44.074495 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bee63-5ad9-4544-845b-aa47f7ba623b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:44.085561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1318/2621] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:43 2022] PUT /identity/v3/projects/63e2df0bf70d4d78b25b720067ab3b49/users/e302188b8b0f4ea596f5020666d9be1a/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:44.096082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1304/2622] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:44.096256 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7234ea-8d18-4216-bf06-223af943b657 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:44.096448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7234ea-8d18-4216-bf06-223af943b657 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:44.096723 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7234ea-8d18-4216-bf06-223af943b657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:44.103775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:44.109276 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18739828-1f2e-4ad1-a405-ea66a7ec26f1 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:44.110704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-18739828-1f2e-4ad1-a405-ea66a7ec26f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:44.128132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ae7234ea-8d18-4216-bf06-223af943b657 None None] MFA Rules not processed for user `e302188b8b0f4ea596f5020666d9be1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:44.129349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ae7234ea-8d18-4216-bf06-223af943b657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:44.169118 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18739828-1f2e-4ad1-a405-ea66a7ec26f1 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:44.204581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1319/2623] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:44.216363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7234ea-8d18-4216-bf06-223af943b657 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:44.217760 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ae7234ea-8d18-4216-bf06-223af943b657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:44.221235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d4df3-60d6-4e69-b5e7-8625efcc32df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:44.221851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d4df3-60d6-4e69-b5e7-8625efcc32df service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:44.222148 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d4df3-60d6-4e69-b5e7-8625efcc32df service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:44.222433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d4df3-60d6-4e69-b5e7-8625efcc32df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:44.229888 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7234ea-8d18-4216-bf06-223af943b657 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:44.233477 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4daef9-eef3-418d-b4e0-9d7ad6482064 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:44.234100 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4daef9-eef3-418d-b4e0-9d7ad6482064 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:44.234406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4daef9-eef3-418d-b4e0-9d7ad6482064 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:44.234673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4daef9-eef3-418d-b4e0-9d7ad6482064 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:44.244087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4daef9-eef3-418d-b4e0-9d7ad6482064 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:44.245502 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4daef9-eef3-418d-b4e0-9d7ad6482064 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:44.255580 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1320/2624] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:16:44.273307 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:44.293218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d4df3-60d6-4e69-b5e7-8625efcc32df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:44.294748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d4df3-60d6-4e69-b5e7-8625efcc32df service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:44.306042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1305/2625] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:44 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:16:44.655696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4daef9-eef3-418d-b4e0-9d7ad6482064 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:44.670781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4daef9-eef3-418d-b4e0-9d7ad6482064 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:44.672827 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d4df3-60d6-4e69-b5e7-8625efcc32df service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:44.676530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b252fa-3e15-4a7a-ad1e-95af80d798ac tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a53eb2d29b74e019acb8e59ef23e8d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '299b3e4ec03947f7ae792f249eb166b5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:44.677296 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b252fa-3e15-4a7a-ad1e-95af80d798ac tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:44.677792 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b252fa-3e15-4a7a-ad1e-95af80d798ac tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:44.678139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b252fa-3e15-4a7a-ad1e-95af80d798ac tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:44.678825 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b252fa-3e15-4a7a-ad1e-95af80d798ac tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:16:44.679128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b252fa-3e15-4a7a-ad1e-95af80d798ac tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:44.680800 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b252fa-3e15-4a7a-ad1e-95af80d798ac tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:44.693647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1321/2626] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:16:44.698237 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b252fa-3e15-4a7a-ad1e-95af80d798ac tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:44.709704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b252fa-3e15-4a7a-ad1e-95af80d798ac tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:44.714364 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63164a0-ae55-44b8-835b-f48af7cc41b2 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a53eb2d29b74e019acb8e59ef23e8d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '299b3e4ec03947f7ae792f249eb166b5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:44.714924 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63164a0-ae55-44b8-835b-f48af7cc41b2 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:44.715315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63164a0-ae55-44b8-835b-f48af7cc41b2 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:44.715623 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63164a0-ae55-44b8-835b-f48af7cc41b2 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:44.716274 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63164a0-ae55-44b8-835b-f48af7cc41b2 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:44.718174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63164a0-ae55-44b8-835b-f48af7cc41b2 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:44.737444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d4df3-60d6-4e69-b5e7-8625efcc32df service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:44.742777 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b8290-a597-4d94-aa58-c469040db0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:44.743414 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b8290-a597-4d94-aa58-c469040db0a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:44.743943 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b8290-a597-4d94-aa58-c469040db0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:44.744219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b8290-a597-4d94-aa58-c469040db0a5 admin admin] PATH_INFO: `/v3/users/a0e11d99fe8a49f7ab6afe58fb07dd6e` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:44.745341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b8290-a597-4d94-aa58-c469040db0a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e11d99fe8a49f7ab6afe58fb07dd6e)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:44.747350 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b8290-a597-4d94-aa58-c469040db0a5 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:44.751587 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1322/2627] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:44 2022] POST /identity/v3/projects => generated 354 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:44.753299 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63164a0-ae55-44b8-835b-f48af7cc41b2 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:44.797281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-736b8290-a597-4d94-aa58-c469040db0a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e11d99fe8a49f7ab6afe58fb07dd6e'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:44.807001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-736b8290-a597-4d94-aa58-c469040db0a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e11d99fe8a49f7ab6afe58fb07dd6e'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:44.820131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-736b8290-a597-4d94-aa58-c469040db0a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e11d99fe8a49f7ab6afe58fb07dd6e'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:44.820531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63164a0-ae55-44b8-835b-f48af7cc41b2 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:44.825949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d640f3-31b1-40e8-8468-b251caa62056 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a53eb2d29b74e019acb8e59ef23e8d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '299b3e4ec03947f7ae792f249eb166b5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:44.826667 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d640f3-31b1-40e8-8468-b251caa62056 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:44.826886 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d640f3-31b1-40e8-8468-b251caa62056 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:44.827159 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d640f3-31b1-40e8-8468-b251caa62056 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:44.827335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-736b8290-a597-4d94-aa58-c469040db0a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e11d99fe8a49f7ab6afe58fb07dd6e'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:44.827829 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d640f3-31b1-40e8-8468-b251caa62056 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:16:44.828012 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-736b8290-a597-4d94-aa58-c469040db0a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e11d99fe8a49f7ab6afe58fb07dd6e'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:44.828137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d640f3-31b1-40e8-8468-b251caa62056 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:44.829802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d640f3-31b1-40e8-8468-b251caa62056 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:44.834012 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1306/2628] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:44 2022] DELETE /identity/v3/users/a0e11d99fe8a49f7ab6afe58fb07dd6e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:44.838391 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8514d1-b0cb-4a1b-973b-d164bf8df137 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:44.838819 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8514d1-b0cb-4a1b-973b-d164bf8df137 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:44.839092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8514d1-b0cb-4a1b-973b-d164bf8df137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:44.844970 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1323/2629] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:16:44.859018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-3c8514d1-b0cb-4a1b-973b-d164bf8df137 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:44.860224 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-3c8514d1-b0cb-4a1b-973b-d164bf8df137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:44.880628 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d640f3-31b1-40e8-8468-b251caa62056 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:44.882640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-62d640f3-31b1-40e8-8468-b251caa62056 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:44.895768 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d640f3-31b1-40e8-8468-b251caa62056 tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:44.926061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1307/2630] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:44.928084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:44.936614 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8514d1-b0cb-4a1b-973b-d164bf8df137 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:44.937981 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-3c8514d1-b0cb-4a1b-973b-d164bf8df137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:44.944607 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:44.956390 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924acdd7-355c-41fa-b5f9-74dc47a2761e tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a53eb2d29b74e019acb8e59ef23e8d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '299b3e4ec03947f7ae792f249eb166b5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:44.957117 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924acdd7-355c-41fa-b5f9-74dc47a2761e tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:44.957471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924acdd7-355c-41fa-b5f9-74dc47a2761e tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:44.957819 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924acdd7-355c-41fa-b5f9-74dc47a2761e tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] PATH_INFO: `/v3/projects/bf5080e2d85d4463b044db9a9e5a9ffc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:44.959073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924acdd7-355c-41fa-b5f9-74dc47a2761e tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bf5080e2d85d4463b044db9a9e5a9ffc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:44.961049 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924acdd7-355c-41fa-b5f9-74dc47a2761e tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:45.000818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8514d1-b0cb-4a1b-973b-d164bf8df137 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:45.010694 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-924acdd7-355c-41fa-b5f9-74dc47a2761e tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf5080e2d85d4463b044db9a9e5a9ffc', 'request_id': 'req-924acdd7-355c-41fa-b5f9-74dc47a2761e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.018941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-924acdd7-355c-41fa-b5f9-74dc47a2761e tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf5080e2d85d4463b044db9a9e5a9ffc', 'request_id': 'req-924acdd7-355c-41fa-b5f9-74dc47a2761e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.025053 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-924acdd7-355c-41fa-b5f9-74dc47a2761e tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf5080e2d85d4463b044db9a9e5a9ffc', 'request_id': 'req-924acdd7-355c-41fa-b5f9-74dc47a2761e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.038217 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-924acdd7-355c-41fa-b5f9-74dc47a2761e tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] The token cache is being invalidate because project bf5080e2d85d4463b044db9a9e5a9ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:45.038607 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-924acdd7-355c-41fa-b5f9-74dc47a2761e tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.039345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-924acdd7-355c-41fa-b5f9-74dc47a2761e tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.041098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1324/2631] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:44 2022] DELETE /identity/v3/projects/bf5080e2d85d4463b044db9a9e5a9ffc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:45.181737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899729a8-147a-41a5-a0c9-edaf78503f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:45.182282 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899729a8-147a-41a5-a0c9-edaf78503f7e service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:45.182407 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899729a8-147a-41a5-a0c9-edaf78503f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:45.182598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899729a8-147a-41a5-a0c9-edaf78503f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:45.184699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-899729a8-147a-41a5-a0c9-edaf78503f7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:45.251843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924acdd7-355c-41fa-b5f9-74dc47a2761e tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:45.253220 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-924acdd7-355c-41fa-b5f9-74dc47a2761e tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:45.258935 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899729a8-147a-41a5-a0c9-edaf78503f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:45.260406 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899729a8-147a-41a5-a0c9-edaf78503f7e service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:45.266533 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924acdd7-355c-41fa-b5f9-74dc47a2761e tempest-VolumeQuotaClassesTest-2007423229 tempest-VolumeQuotaClassesTest-2007423229-project-admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:45.272014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfa61b-53f3-4160-8eab-6ac6c2c4d121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:45.272554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1308/2632] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:16:45.272720 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfa61b-53f3-4160-8eab-6ac6c2c4d121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:45.272906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfa61b-53f3-4160-8eab-6ac6c2c4d121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:45.273178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfa61b-53f3-4160-8eab-6ac6c2c4d121 admin admin] PATH_INFO: `/v3/users/9c7c1ce6f5044db4acde4159404bf4d6` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:45.274425 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfa61b-53f3-4160-8eab-6ac6c2c4d121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c7c1ce6f5044db4acde4159404bf4d6)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:45.276332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfa61b-53f3-4160-8eab-6ac6c2c4d121 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:45.325750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c5dfa61b-53f3-4160-8eab-6ac6c2c4d121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c7c1ce6f5044db4acde4159404bf4d6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.326812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c5dfa61b-53f3-4160-8eab-6ac6c2c4d121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c7c1ce6f5044db4acde4159404bf4d6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.335723 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c5dfa61b-53f3-4160-8eab-6ac6c2c4d121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c7c1ce6f5044db4acde4159404bf4d6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.343742 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c5dfa61b-53f3-4160-8eab-6ac6c2c4d121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c7c1ce6f5044db4acde4159404bf4d6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.357402 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c5dfa61b-53f3-4160-8eab-6ac6c2c4d121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c7c1ce6f5044db4acde4159404bf4d6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.364025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1325/2633] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:45 2022] DELETE /identity/v3/users/9c7c1ce6f5044db4acde4159404bf4d6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:45.368824 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11875b-d7fe-463d-a392-a6d2505ca10b None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:45.369172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11875b-d7fe-463d-a392-a6d2505ca10b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:45.369450 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11875b-d7fe-463d-a392-a6d2505ca10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:45.389108 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-8f11875b-d7fe-463d-a392-a6d2505ca10b None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:45.390360 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-8f11875b-d7fe-463d-a392-a6d2505ca10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:45.456069 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1309/2634] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:45.465736 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfa61b-53f3-4160-8eab-6ac6c2c4d121 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:45.467301 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c5dfa61b-53f3-4160-8eab-6ac6c2c4d121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:45.509089 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:45.530080 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfa61b-53f3-4160-8eab-6ac6c2c4d121 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:45.534990 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cce208-5647-4731-8275-1eea9738440f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:45.535772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cce208-5647-4731-8275-1eea9738440f service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:45.536086 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cce208-5647-4731-8275-1eea9738440f service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:45.536383 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cce208-5647-4731-8275-1eea9738440f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:45.547082 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cce208-5647-4731-8275-1eea9738440f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:45.548123 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11875b-d7fe-463d-a392-a6d2505ca10b None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:45.549001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cce208-5647-4731-8275-1eea9738440f service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:45.549787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-8f11875b-d7fe-463d-a392-a6d2505ca10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:45.561477 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1326/2635] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:45.562399 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11875b-d7fe-463d-a392-a6d2505ca10b None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:45.567797 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695edd8-4faf-4814-88bf-23f3110eab18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:45.568439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695edd8-4faf-4814-88bf-23f3110eab18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:45.568716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695edd8-4faf-4814-88bf-23f3110eab18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:45.569022 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695edd8-4faf-4814-88bf-23f3110eab18 admin admin] PATH_INFO: `/v3/projects/6c51a839a6994b19b470af006a05f8d7` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:45.570256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695edd8-4faf-4814-88bf-23f3110eab18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c51a839a6994b19b470af006a05f8d7)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:45.572165 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695edd8-4faf-4814-88bf-23f3110eab18 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:45.579399 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:45.623939 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b695edd8-4faf-4814-88bf-23f3110eab18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c51a839a6994b19b470af006a05f8d7', 'request_id': 'req-b695edd8-4faf-4814-88bf-23f3110eab18'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.634458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b695edd8-4faf-4814-88bf-23f3110eab18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c51a839a6994b19b470af006a05f8d7', 'request_id': 'req-b695edd8-4faf-4814-88bf-23f3110eab18'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.640902 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b695edd8-4faf-4814-88bf-23f3110eab18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c51a839a6994b19b470af006a05f8d7', 'request_id': 'req-b695edd8-4faf-4814-88bf-23f3110eab18'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.653576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b695edd8-4faf-4814-88bf-23f3110eab18 admin admin] The token cache is being invalidate because project 6c51a839a6994b19b470af006a05f8d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:45.653871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b695edd8-4faf-4814-88bf-23f3110eab18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.654566 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-b695edd8-4faf-4814-88bf-23f3110eab18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.656071 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1310/2636] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:45 2022] DELETE /identity/v3/projects/6c51a839a6994b19b470af006a05f8d7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:45.660169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cce208-5647-4731-8275-1eea9738440f service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:45.670553 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cce208-5647-4731-8275-1eea9738440f service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:45.743912 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cf477-ae42-4a5a-b678-2c05544c50a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:45.744704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cf477-ae42-4a5a-b678-2c05544c50a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:45.744981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cf477-ae42-4a5a-b678-2c05544c50a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:45.745276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cf477-ae42-4a5a-b678-2c05544c50a2 admin admin] PATH_INFO: `/v3/users/e302188b8b0f4ea596f5020666d9be1a` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:45.746469 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cf477-ae42-4a5a-b678-2c05544c50a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e302188b8b0f4ea596f5020666d9be1a)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:45.748373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cf477-ae42-4a5a-b678-2c05544c50a2 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:45.798987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c89cf477-ae42-4a5a-b678-2c05544c50a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e302188b8b0f4ea596f5020666d9be1a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.800285 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c89cf477-ae42-4a5a-b678-2c05544c50a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e302188b8b0f4ea596f5020666d9be1a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.808261 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c89cf477-ae42-4a5a-b678-2c05544c50a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e302188b8b0f4ea596f5020666d9be1a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.816913 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c89cf477-ae42-4a5a-b678-2c05544c50a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e302188b8b0f4ea596f5020666d9be1a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.829266 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c89cf477-ae42-4a5a-b678-2c05544c50a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e302188b8b0f4ea596f5020666d9be1a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:45.837798 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1327/2637] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:45 2022] DELETE /identity/v3/users/e302188b8b0f4ea596f5020666d9be1a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:45.869427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695edd8-4faf-4814-88bf-23f3110eab18 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:45.870968 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b695edd8-4faf-4814-88bf-23f3110eab18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:45.884762 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695edd8-4faf-4814-88bf-23f3110eab18 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:45.951465 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f22c01-5dc0-47d5-801f-b36716e24985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:45.952134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f22c01-5dc0-47d5-801f-b36716e24985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:45.952415 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f22c01-5dc0-47d5-801f-b36716e24985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:45.952689 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f22c01-5dc0-47d5-801f-b36716e24985 admin admin] PATH_INFO: `/v3/projects/9ea9e1ab8e1741399df95f93f2ba2a73` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:45.953844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f22c01-5dc0-47d5-801f-b36716e24985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ea9e1ab8e1741399df95f93f2ba2a73)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:45.956417 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f22c01-5dc0-47d5-801f-b36716e24985 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:46.018625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-35f22c01-5dc0-47d5-801f-b36716e24985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ea9e1ab8e1741399df95f93f2ba2a73', 'request_id': 'req-35f22c01-5dc0-47d5-801f-b36716e24985'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.028131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-35f22c01-5dc0-47d5-801f-b36716e24985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ea9e1ab8e1741399df95f93f2ba2a73', 'request_id': 'req-35f22c01-5dc0-47d5-801f-b36716e24985'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.035244 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-35f22c01-5dc0-47d5-801f-b36716e24985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ea9e1ab8e1741399df95f93f2ba2a73', 'request_id': 'req-35f22c01-5dc0-47d5-801f-b36716e24985'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.048733 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-35f22c01-5dc0-47d5-801f-b36716e24985 admin admin] The token cache is being invalidate because project 9ea9e1ab8e1741399df95f93f2ba2a73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:46.049065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-35f22c01-5dc0-47d5-801f-b36716e24985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.049966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-35f22c01-5dc0-47d5-801f-b36716e24985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.051630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1311/2638] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:45 2022] DELETE /identity/v3/projects/9ea9e1ab8e1741399df95f93f2ba2a73 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:46.056098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cf477-ae42-4a5a-b678-2c05544c50a2 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:46.057837 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c89cf477-ae42-4a5a-b678-2c05544c50a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:46.071088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cf477-ae42-4a5a-b678-2c05544c50a2 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:46.125960 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:46.133455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facf482-a031-4027-93b3-455d7b05c397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:46.134625 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facf482-a031-4027-93b3-455d7b05c397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.135739 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facf482-a031-4027-93b3-455d7b05c397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.136027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facf482-a031-4027-93b3-455d7b05c397 admin admin] PATH_INFO: `/v3/users/6a53eb2d29b74e019acb8e59ef23e8d5` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.137519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4facf482-a031-4027-93b3-455d7b05c397 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a53eb2d29b74e019acb8e59ef23e8d5)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:46.139508 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4facf482-a031-4027-93b3-455d7b05c397 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:46.177118 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f22c01-5dc0-47d5-801f-b36716e24985 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:46.178844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-35f22c01-5dc0-47d5-801f-b36716e24985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:46.196268 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f22c01-5dc0-47d5-801f-b36716e24985 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:46.200207 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96a7e0-8f7f-462e-966d-d79eb6254d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:46.200920 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96a7e0-8f7f-462e-966d-d79eb6254d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.201377 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96a7e0-8f7f-462e-966d-d79eb6254d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.202095 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96a7e0-8f7f-462e-966d-d79eb6254d32 admin admin] PATH_INFO: `/v3/projects/63e2df0bf70d4d78b25b720067ab3b49` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.203251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96a7e0-8f7f-462e-966d-d79eb6254d32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63e2df0bf70d4d78b25b720067ab3b49)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:46.203896 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4facf482-a031-4027-93b3-455d7b05c397 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a53eb2d29b74e019acb8e59ef23e8d5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.205114 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4facf482-a031-4027-93b3-455d7b05c397 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a53eb2d29b74e019acb8e59ef23e8d5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.205674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96a7e0-8f7f-462e-966d-d79eb6254d32 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:46.213608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4facf482-a031-4027-93b3-455d7b05c397 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a53eb2d29b74e019acb8e59ef23e8d5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.223388 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4facf482-a031-4027-93b3-455d7b05c397 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a53eb2d29b74e019acb8e59ef23e8d5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.229550 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:46.237855 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4facf482-a031-4027-93b3-455d7b05c397 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a53eb2d29b74e019acb8e59ef23e8d5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.245459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1328/2639] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:46 2022] DELETE /identity/v3/users/6a53eb2d29b74e019acb8e59ef23e8d5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:46.258999 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-da96a7e0-8f7f-462e-966d-d79eb6254d32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63e2df0bf70d4d78b25b720067ab3b49', 'request_id': 'req-da96a7e0-8f7f-462e-966d-d79eb6254d32'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.266853 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-da96a7e0-8f7f-462e-966d-d79eb6254d32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63e2df0bf70d4d78b25b720067ab3b49', 'request_id': 'req-da96a7e0-8f7f-462e-966d-d79eb6254d32'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.274256 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-da96a7e0-8f7f-462e-966d-d79eb6254d32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63e2df0bf70d4d78b25b720067ab3b49', 'request_id': 'req-da96a7e0-8f7f-462e-966d-d79eb6254d32'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.294627 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-da96a7e0-8f7f-462e-966d-d79eb6254d32 admin admin] The token cache is being invalidate because project 63e2df0bf70d4d78b25b720067ab3b49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:46.294925 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-da96a7e0-8f7f-462e-966d-d79eb6254d32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.295622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-da96a7e0-8f7f-462e-966d-d79eb6254d32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.297171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1312/2640] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:46 2022] DELETE /identity/v3/projects/63e2df0bf70d4d78b25b720067ab3b49 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:46.302716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886d61f-3264-475c-98c2-4446d2e4369f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.302906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886d61f-3264-475c-98c2-4446d2e4369f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.303185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886d61f-3264-475c-98c2-4446d2e4369f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.334292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f886d61f-3264-475c-98c2-4446d2e4369f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:46.335639 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f886d61f-3264-475c-98c2-4446d2e4369f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:46.419672 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1329/2641] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:46.427523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689ddd-604a-4a45-93e5-40962190f3b8 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.427807 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689ddd-604a-4a45-93e5-40962190f3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.428113 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689ddd-604a-4a45-93e5-40962190f3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.448054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d9689ddd-604a-4a45-93e5-40962190f3b8 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:46.449251 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d9689ddd-604a-4a45-93e5-40962190f3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:46.457811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1313/2642] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:46.462021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886d61f-3264-475c-98c2-4446d2e4369f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:46.471941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886d61f-3264-475c-98c2-4446d2e4369f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:46.476641 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38997e54-ba33-4066-88d7-52cbfa666f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:46.477180 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38997e54-ba33-4066-88d7-52cbfa666f2c admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.477455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38997e54-ba33-4066-88d7-52cbfa666f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.477719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38997e54-ba33-4066-88d7-52cbfa666f2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.478405 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38997e54-ba33-4066-88d7-52cbfa666f2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:16:46.478706 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38997e54-ba33-4066-88d7-52cbfa666f2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:46.480292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38997e54-ba33-4066-88d7-52cbfa666f2c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:46.493079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1330/2643] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:16:46.497682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9689ddd-604a-4a45-93e5-40962190f3b8 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:46.507625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9689ddd-604a-4a45-93e5-40962190f3b8 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:46.512342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609018d0-0317-45fa-ab36-891aeb95b9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:46.512941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609018d0-0317-45fa-ab36-891aeb95b9b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.513221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609018d0-0317-45fa-ab36-891aeb95b9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.513489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609018d0-0317-45fa-ab36-891aeb95b9b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.514214 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609018d0-0317-45fa-ab36-891aeb95b9b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:46.516051 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609018d0-0317-45fa-ab36-891aeb95b9b5 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:46.540917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38997e54-ba33-4066-88d7-52cbfa666f2c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:46.542927 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-38997e54-ba33-4066-88d7-52cbfa666f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:46.549017 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1314/2644] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:46 2022] POST /identity/v3/projects => generated 409 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:46.551086 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609018d0-0317-45fa-ab36-891aeb95b9b5 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:46.556995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38997e54-ba33-4066-88d7-52cbfa666f2c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:46.622652 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e13fe-c8a3-45c9-8d91-4c04ad46cc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:46.623318 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e13fe-c8a3-45c9-8d91-4c04ad46cc6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.623591 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e13fe-c8a3-45c9-8d91-4c04ad46cc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.624011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e13fe-c8a3-45c9-8d91-4c04ad46cc6b admin admin] PATH_INFO: `/v3/projects/299b3e4ec03947f7ae792f249eb166b5` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.625155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e13fe-c8a3-45c9-8d91-4c04ad46cc6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=299b3e4ec03947f7ae792f249eb166b5)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:46.625315 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609018d0-0317-45fa-ab36-891aeb95b9b5 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:46.627954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e13fe-c8a3-45c9-8d91-4c04ad46cc6b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:46.635307 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5490f8-d3aa-4949-a3d6-deed10afa663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:46.635924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5490f8-d3aa-4949-a3d6-deed10afa663 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.636200 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5490f8-d3aa-4949-a3d6-deed10afa663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.636466 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5490f8-d3aa-4949-a3d6-deed10afa663 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.637086 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:46.637208 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5490f8-d3aa-4949-a3d6-deed10afa663 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:46.639218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5490f8-d3aa-4949-a3d6-deed10afa663 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:46.662812 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1315/2645] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:46 2022] POST /identity/v3/users => generated 305 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:46.667190 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5490f8-d3aa-4949-a3d6-deed10afa663 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:46.677715 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5490f8-d3aa-4949-a3d6-deed10afa663 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:46.691254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-583e13fe-c8a3-45c9-8d91-4c04ad46cc6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '299b3e4ec03947f7ae792f249eb166b5', 'request_id': 'req-583e13fe-c8a3-45c9-8d91-4c04ad46cc6b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.699993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-583e13fe-c8a3-45c9-8d91-4c04ad46cc6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '299b3e4ec03947f7ae792f249eb166b5', 'request_id': 'req-583e13fe-c8a3-45c9-8d91-4c04ad46cc6b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.706296 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-583e13fe-c8a3-45c9-8d91-4c04ad46cc6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '299b3e4ec03947f7ae792f249eb166b5', 'request_id': 'req-583e13fe-c8a3-45c9-8d91-4c04ad46cc6b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.717563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-583e13fe-c8a3-45c9-8d91-4c04ad46cc6b admin admin] The token cache is being invalidate because project 299b3e4ec03947f7ae792f249eb166b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:46.717851 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-583e13fe-c8a3-45c9-8d91-4c04ad46cc6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.718515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-583e13fe-c8a3-45c9-8d91-4c04ad46cc6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:46.720025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1331/2646] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:46 2022] DELETE /identity/v3/projects/299b3e4ec03947f7ae792f249eb166b5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:46.724686 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f4cc0-75b1-42b4-877a-939a8b181dc8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.725129 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f4cc0-75b1-42b4-877a-939a8b181dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.725493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f4cc0-75b1-42b4-877a-939a8b181dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.739480 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d59d2e-3219-4212-a620-ccc0871363b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:46.740010 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d59d2e-3219-4212-a620-ccc0871363b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.740283 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d59d2e-3219-4212-a620-ccc0871363b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.740579 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d59d2e-3219-4212-a620-ccc0871363b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.741219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d59d2e-3219-4212-a620-ccc0871363b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:46.742845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d59d2e-3219-4212-a620-ccc0871363b6 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:46.748596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d58f4cc0-75b1-42b4-877a-939a8b181dc8 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:46.749822 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d58f4cc0-75b1-42b4-877a-939a8b181dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:46.754434 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1316/2647] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:46.758345 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d59d2e-3219-4212-a620-ccc0871363b6 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:46.759010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1332/2648] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:46.760010 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-62d59d2e-3219-4212-a620-ccc0871363b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:46.767484 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3c141-40ab-4624-95bd-2ab0d82df849 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.767764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3c141-40ab-4624-95bd-2ab0d82df849 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.768047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3c141-40ab-4624-95bd-2ab0d82df849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.769013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:46.772907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d59d2e-3219-4212-a620-ccc0871363b6 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:46.780534 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016df74-aa10-4db7-9792-ded4c304787a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:46.781150 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016df74-aa10-4db7-9792-ded4c304787a admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.781548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016df74-aa10-4db7-9792-ded4c304787a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.781875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016df74-aa10-4db7-9792-ded4c304787a admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.782718 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016df74-aa10-4db7-9792-ded4c304787a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:46.784359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016df74-aa10-4db7-9792-ded4c304787a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:46.788754 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d2c3c141-40ab-4624-95bd-2ab0d82df849 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:46.789953 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d2c3c141-40ab-4624-95bd-2ab0d82df849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:46.795580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1317/2649] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:16:46.799697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1333/2650] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:46.800383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016df74-aa10-4db7-9792-ded4c304787a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:46.803010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3c141-40ab-4624-95bd-2ab0d82df849 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:46.811873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016df74-aa10-4db7-9792-ded4c304787a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:46.813067 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3c141-40ab-4624-95bd-2ab0d82df849 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:46.817388 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab236cc5-f3a9-4699-8a95-8d0faa3d236b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:46.818036 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab236cc5-f3a9-4699-8a95-8d0faa3d236b admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.818422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87450b01-702a-4c10-82c8-a6ee522a8194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:46.818529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab236cc5-f3a9-4699-8a95-8d0faa3d236b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.818630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab236cc5-f3a9-4699-8a95-8d0faa3d236b admin admin] PATH_INFO: `/v3/projects/47d2a1134608495d85e43e18995c8acc/users/fde5f401fed8452dadee6d5f581ad57f/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.818926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87450b01-702a-4c10-82c8-a6ee522a8194 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.819204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87450b01-702a-4c10-82c8-a6ee522a8194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.819479 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87450b01-702a-4c10-82c8-a6ee522a8194 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.820133 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87450b01-702a-4c10-82c8-a6ee522a8194 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:16:46.820432 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87450b01-702a-4c10-82c8-a6ee522a8194 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:46.822021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87450b01-702a-4c10-82c8-a6ee522a8194 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:46.834433 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab236cc5-f3a9-4699-8a95-8d0faa3d236b admin admin] RBAC: Authorizing `identity:create_grant(project_id=47d2a1134608495d85e43e18995c8acc, user_id=fde5f401fed8452dadee6d5f581ad57f, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:46.834728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1334/2651] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:16:46.838967 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87450b01-702a-4c10-82c8-a6ee522a8194 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:46.840183 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab236cc5-f3a9-4699-8a95-8d0faa3d236b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:46.849472 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87450b01-702a-4c10-82c8-a6ee522a8194 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:46.854722 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1318/2652] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:46 2022] PUT /identity/v3/projects/47d2a1134608495d85e43e18995c8acc/users/fde5f401fed8452dadee6d5f581ad57f/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:46.866775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffaec6c-38ff-455f-83b6-9cb86baae7ce None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.867064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffaec6c-38ff-455f-83b6-9cb86baae7ce None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.867338 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffaec6c-38ff-455f-83b6-9cb86baae7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.897784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-2ffaec6c-38ff-455f-83b6-9cb86baae7ce None None] MFA Rules not processed for user `fde5f401fed8452dadee6d5f581ad57f`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:46.899016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2ffaec6c-38ff-455f-83b6-9cb86baae7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:46.911921 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5721225-0807-4565-a05e-b5e9f6e1fa51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:46.912534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5721225-0807-4565-a05e-b5e9f6e1fa51 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:46.912814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5721225-0807-4565-a05e-b5e9f6e1fa51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:46.913088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5721225-0807-4565-a05e-b5e9f6e1fa51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:46.913733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5721225-0807-4565-a05e-b5e9f6e1fa51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:46.915652 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5721225-0807-4565-a05e-b5e9f6e1fa51 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:46.947508 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1335/2653] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:46 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:46.948495 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5721225-0807-4565-a05e-b5e9f6e1fa51 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:46.965672 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1319/2654] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:46 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:46.970153 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffaec6c-38ff-455f-83b6-9cb86baae7ce None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:46.980578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffaec6c-38ff-455f-83b6-9cb86baae7ce None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.003058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:47.013591 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5721225-0807-4565-a05e-b5e9f6e1fa51 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.019660 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c4418-31ac-4b8a-8bb2-87e094e1795a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.020485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c4418-31ac-4b8a-8bb2-87e094e1795a admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.020765 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c4418-31ac-4b8a-8bb2-87e094e1795a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.021044 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c4418-31ac-4b8a-8bb2-87e094e1795a admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.021679 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c4418-31ac-4b8a-8bb2-87e094e1795a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.023621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c4418-31ac-4b8a-8bb2-87e094e1795a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.039355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc15ed-5d48-418c-b7fd-7b1dbdc02010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.039978 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc15ed-5d48-418c-b7fd-7b1dbdc02010 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.040264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc15ed-5d48-418c-b7fd-7b1dbdc02010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.040787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc15ed-5d48-418c-b7fd-7b1dbdc02010 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.041153 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc15ed-5d48-418c-b7fd-7b1dbdc02010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.043082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc15ed-5d48-418c-b7fd-7b1dbdc02010 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.045455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1336/2655] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:46 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:47.049979 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c4418-31ac-4b8a-8bb2-87e094e1795a admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.059664 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c4418-31ac-4b8a-8bb2-87e094e1795a admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.065520 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f73ab-603a-4937-bd58-5ea5b44d165c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.066147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f73ab-603a-4937-bd58-5ea5b44d165c admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.066427 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f73ab-603a-4937-bd58-5ea5b44d165c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.066703 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f73ab-603a-4937-bd58-5ea5b44d165c admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.067423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f73ab-603a-4937-bd58-5ea5b44d165c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.069151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f73ab-603a-4937-bd58-5ea5b44d165c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.071633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1320/2656] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:46 2022] POST /identity/v3/projects => generated 407 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:47.075795 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc15ed-5d48-418c-b7fd-7b1dbdc02010 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.079601 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1337/2657] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:16:47.083448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f73ab-603a-4937-bd58-5ea5b44d165c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.094343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f73ab-603a-4937-bd58-5ea5b44d165c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.124948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:47.142618 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc15ed-5d48-418c-b7fd-7b1dbdc02010 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.147910 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09296987-b2ff-485d-9a62-f6999e269fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.148511 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09296987-b2ff-485d-9a62-f6999e269fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.148790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09296987-b2ff-485d-9a62-f6999e269fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.149057 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09296987-b2ff-485d-9a62-f6999e269fb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.149679 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09296987-b2ff-485d-9a62-f6999e269fb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.151625 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09296987-b2ff-485d-9a62-f6999e269fb9 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.153077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc378a-bfc5-44e0-b833-a021cc537acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.153684 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc378a-bfc5-44e0-b833-a021cc537acb admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.153965 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc378a-bfc5-44e0-b833-a021cc537acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.154265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc378a-bfc5-44e0-b833-a021cc537acb admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.154912 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc378a-bfc5-44e0-b833-a021cc537acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.156488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc378a-bfc5-44e0-b833-a021cc537acb admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.167372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1338/2658] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:47.171595 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc378a-bfc5-44e0-b833-a021cc537acb admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.176310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1321/2659] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:47 2022] POST /identity/v3/users => generated 310 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:47.180103 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09296987-b2ff-485d-9a62-f6999e269fb9 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.184276 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc378a-bfc5-44e0-b833-a021cc537acb admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.189107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aecebd-1425-4da6-97fa-696a4b472589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.189759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aecebd-1425-4da6-97fa-696a4b472589 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.190038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aecebd-1425-4da6-97fa-696a4b472589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.190347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aecebd-1425-4da6-97fa-696a4b472589 admin admin] PATH_INFO: `/v3/projects/a1fdc67aa6b94d6f9772d3435cdda1a0/users/3a9794edf60e4190ade0ac7416f88bf7/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.191112 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09296987-b2ff-485d-9a62-f6999e269fb9 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.195973 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a133571-4fdc-4cc0-a50d-c1639f3bce38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.196494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a133571-4fdc-4cc0-a50d-c1639f3bce38 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.197014 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a133571-4fdc-4cc0-a50d-c1639f3bce38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.197014 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a133571-4fdc-4cc0-a50d-c1639f3bce38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.197670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a133571-4fdc-4cc0-a50d-c1639f3bce38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.199314 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a133571-4fdc-4cc0-a50d-c1639f3bce38 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.206628 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aecebd-1425-4da6-97fa-696a4b472589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1fdc67aa6b94d6f9772d3435cdda1a0, user_id=3a9794edf60e4190ade0ac7416f88bf7, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.209995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aecebd-1425-4da6-97fa-696a4b472589 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.210740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1322/2660] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:16:47.215394 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a133571-4fdc-4cc0-a50d-c1639f3bce38 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.225987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a133571-4fdc-4cc0-a50d-c1639f3bce38 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.228497 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1339/2661] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:47 2022] PUT /identity/v3/projects/a1fdc67aa6b94d6f9772d3435cdda1a0/users/3a9794edf60e4190ade0ac7416f88bf7/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:47.239903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5902e08-16f3-4cfa-8402-574cd14a102d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.240186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5902e08-16f3-4cfa-8402-574cd14a102d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.240467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5902e08-16f3-4cfa-8402-574cd14a102d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.269489 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f5902e08-16f3-4cfa-8402-574cd14a102d None None] MFA Rules not processed for user `3a9794edf60e4190ade0ac7416f88bf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:47.270722 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f5902e08-16f3-4cfa-8402-574cd14a102d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:47.288296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6629dcb-1f16-4517-aee7-e5a30e2e7eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.288967 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6629dcb-1f16-4517-aee7-e5a30e2e7eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.289247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6629dcb-1f16-4517-aee7-e5a30e2e7eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.289519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6629dcb-1f16-4517-aee7-e5a30e2e7eaa admin admin] PATH_INFO: `/v3/projects/6f09a326bf644d918828a5b9a554c5c5/users/07ca58087fc242a6a8b8ea57d90b3cf9/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.305907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6629dcb-1f16-4517-aee7-e5a30e2e7eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f09a326bf644d918828a5b9a554c5c5, user_id=07ca58087fc242a6a8b8ea57d90b3cf9, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.308645 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6629dcb-1f16-4517-aee7-e5a30e2e7eaa admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.321666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1323/2662] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:47 2022] PUT /identity/v3/projects/6f09a326bf644d918828a5b9a554c5c5/users/07ca58087fc242a6a8b8ea57d90b3cf9/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:47.326061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6629dcb-1f16-4517-aee7-e5a30e2e7eaa admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.338174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1340/2663] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:47.342574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5902e08-16f3-4cfa-8402-574cd14a102d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.353550 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5902e08-16f3-4cfa-8402-574cd14a102d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.388047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:47.392038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6629dcb-1f16-4517-aee7-e5a30e2e7eaa admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.396953 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f182bd5-ec8b-4275-a22f-063f80dfbdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.397553 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f182bd5-ec8b-4275-a22f-063f80dfbdf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.397829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f182bd5-ec8b-4275-a22f-063f80dfbdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.398176 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f182bd5-ec8b-4275-a22f-063f80dfbdf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.398767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f182bd5-ec8b-4275-a22f-063f80dfbdf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.400337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f182bd5-ec8b-4275-a22f-063f80dfbdf8 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.410988 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1324/2664] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:16:47.415126 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f182bd5-ec8b-4275-a22f-063f80dfbdf8 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.420499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eaf1d4-9541-4e5e-8b3a-2e8f7c68c2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.421111 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaf1d4-9541-4e5e-8b3a-2e8f7c68c2d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.421581 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaf1d4-9541-4e5e-8b3a-2e8f7c68c2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.421861 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaf1d4-9541-4e5e-8b3a-2e8f7c68c2d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.422538 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eaf1d4-9541-4e5e-8b3a-2e8f7c68c2d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.424989 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eaf1d4-9541-4e5e-8b3a-2e8f7c68c2d5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.425147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f182bd5-ec8b-4275-a22f-063f80dfbdf8 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.429741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6803554-499d-4771-8c4b-4bd7cae0d5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.430426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6803554-499d-4771-8c4b-4bd7cae0d5fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.430700 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6803554-499d-4771-8c4b-4bd7cae0d5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.430966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6803554-499d-4771-8c4b-4bd7cae0d5fc admin admin] PATH_INFO: `/v3/projects/6f09a326bf644d918828a5b9a554c5c5/users/07ca58087fc242a6a8b8ea57d90b3cf9/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.432843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6803554-499d-4771-8c4b-4bd7cae0d5fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f09a326bf644d918828a5b9a554c5c5, user_id=07ca58087fc242a6a8b8ea57d90b3cf9, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.435476 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6803554-499d-4771-8c4b-4bd7cae0d5fc admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.445768 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1325/2665] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:47 2022] PUT /identity/v3/projects/6f09a326bf644d918828a5b9a554c5c5/users/07ca58087fc242a6a8b8ea57d90b3cf9/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:47.456468 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5efac-9c4c-44e9-bc8d-a388d572465f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.456755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5efac-9c4c-44e9-bc8d-a388d572465f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.457041 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5efac-9c4c-44e9-bc8d-a388d572465f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.459446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1341/2666] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:47 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:47.463241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eaf1d4-9541-4e5e-8b3a-2e8f7c68c2d5 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.486491 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a7f5efac-9c4c-44e9-bc8d-a388d572465f None None] MFA Rules not processed for user `07ca58087fc242a6a8b8ea57d90b3cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:47.487787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a7f5efac-9c4c-44e9-bc8d-a388d572465f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:47.504589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:47.529229 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eaf1d4-9541-4e5e-8b3a-2e8f7c68c2d5 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.534503 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e9d5b-f57b-4e7e-a7ad-710e1ff3b4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.535110 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e9d5b-f57b-4e7e-a7ad-710e1ff3b4ea admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.535414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e9d5b-f57b-4e7e-a7ad-710e1ff3b4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.535656 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e9d5b-f57b-4e7e-a7ad-710e1ff3b4ea admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.536292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e9d5b-f57b-4e7e-a7ad-710e1ff3b4ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.538220 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e9d5b-f57b-4e7e-a7ad-710e1ff3b4ea admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.558564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1326/2667] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:47 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:47.559998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1342/2668] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:47 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:47.563758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5efac-9c4c-44e9-bc8d-a388d572465f None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.569963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e9d5b-f57b-4e7e-a7ad-710e1ff3b4ea admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.571774 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3a0e9d5b-f57b-4e7e-a7ad-710e1ff3b4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:47.574583 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5efac-9c4c-44e9-bc8d-a388d572465f None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.579430 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac94931-c97d-4c7c-90f3-73423699ab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.580608 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac94931-c97d-4c7c-90f3-73423699ab59 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.580878 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac94931-c97d-4c7c-90f3-73423699ab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.581139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac94931-c97d-4c7c-90f3-73423699ab59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.581774 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac94931-c97d-4c7c-90f3-73423699ab59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.583547 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac94931-c97d-4c7c-90f3-73423699ab59 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.593724 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1327/2669] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:47.598244 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac94931-c97d-4c7c-90f3-73423699ab59 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.608606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac94931-c97d-4c7c-90f3-73423699ab59 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.614287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64171cdd-1b4c-45f3-ba1d-4d80549903e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.614916 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64171cdd-1b4c-45f3-ba1d-4d80549903e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.615195 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64171cdd-1b4c-45f3-ba1d-4d80549903e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.615469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64171cdd-1b4c-45f3-ba1d-4d80549903e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.616110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64171cdd-1b4c-45f3-ba1d-4d80549903e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.617696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64171cdd-1b4c-45f3-ba1d-4d80549903e5 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.628763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1328/2670] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:16:47.632812 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e9d5b-f57b-4e7e-a7ad-710e1ff3b4ea admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.633226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64171cdd-1b4c-45f3-ba1d-4d80549903e5 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.636422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2824c-c154-4e1d-afbf-4bb74e0043a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.637027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2824c-c154-4e1d-afbf-4bb74e0043a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.637296 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2824c-c154-4e1d-afbf-4bb74e0043a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.637561 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2824c-c154-4e1d-afbf-4bb74e0043a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.643235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64171cdd-1b4c-45f3-ba1d-4d80549903e5 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.648030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2824c-c154-4e1d-afbf-4bb74e0043a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.648163 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc8f0d-db69-422a-95bf-ee628311f0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.648692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc8f0d-db69-422a-95bf-ee628311f0f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.648969 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc8f0d-db69-422a-95bf-ee628311f0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.649240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc8f0d-db69-422a-95bf-ee628311f0f4 admin admin] PATH_INFO: `/v3/projects/91579a604c914c6cb7c4df078ef8dcd6/users/bf35d7be8f75409eb92d7d508f5316ac/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.649483 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2824c-c154-4e1d-afbf-4bb74e0043a8 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.659151 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1343/2671] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:47 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:16:47.666202 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc8f0d-db69-422a-95bf-ee628311f0f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91579a604c914c6cb7c4df078ef8dcd6, user_id=bf35d7be8f75409eb92d7d508f5316ac, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.669441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc8f0d-db69-422a-95bf-ee628311f0f4 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.684598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1329/2672] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:47 2022] PUT /identity/v3/projects/91579a604c914c6cb7c4df078ef8dcd6/users/bf35d7be8f75409eb92d7d508f5316ac/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:47.697166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9cb81-61f4-4929-94c0-0bd39e0e4fd8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.697450 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9cb81-61f4-4929-94c0-0bd39e0e4fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.697747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9cb81-61f4-4929-94c0-0bd39e0e4fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.727106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-47b9cb81-61f4-4929-94c0-0bd39e0e4fd8 None None] MFA Rules not processed for user `bf35d7be8f75409eb92d7d508f5316ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:47.728661 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-47b9cb81-61f4-4929-94c0-0bd39e0e4fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:47.781515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:47.813700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1344/2673] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:47.818826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc8f0d-db69-422a-95bf-ee628311f0f4 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.839920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9cb81-61f4-4929-94c0-0bd39e0e4fd8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.886781 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc8f0d-db69-422a-95bf-ee628311f0f4 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.891514 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad849e-5bfc-494f-b3ee-b7fd615edb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.892147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad849e-5bfc-494f-b3ee-b7fd615edb47 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.892421 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad849e-5bfc-494f-b3ee-b7fd615edb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.892686 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad849e-5bfc-494f-b3ee-b7fd615edb47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.893311 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad849e-5bfc-494f-b3ee-b7fd615edb47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.895261 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad849e-5bfc-494f-b3ee-b7fd615edb47 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.899504 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9cb81-61f4-4929-94c0-0bd39e0e4fd8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.903232 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65980418-77c8-485a-9ae0-7f361fbdfab0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.903833 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65980418-77c8-485a-9ae0-7f361fbdfab0 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.904116 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65980418-77c8-485a-9ae0-7f361fbdfab0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:47.904382 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65980418-77c8-485a-9ae0-7f361fbdfab0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:47.922417 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1330/2674] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:47 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:47.927095 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad849e-5bfc-494f-b3ee-b7fd615edb47 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:47.963439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65980418-77c8-485a-9ae0-7f361fbdfab0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:47.964865 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65980418-77c8-485a-9ae0-7f361fbdfab0 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:47.974686 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1345/2675] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:47 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:16:47.980987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:47.993847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad849e-5bfc-494f-b3ee-b7fd615edb47 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:47.998847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332688c-e29c-4006-b796-b5aa6eaacf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:47.999525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332688c-e29c-4006-b796-b5aa6eaacf0c admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:47.999876 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332688c-e29c-4006-b796-b5aa6eaacf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:48.000218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332688c-e29c-4006-b796-b5aa6eaacf0c admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:48.000924 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332688c-e29c-4006-b796-b5aa6eaacf0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:48.002966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332688c-e29c-4006-b796-b5aa6eaacf0c admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:48.024359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1331/2676] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:47 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:48.028435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65980418-77c8-485a-9ae0-7f361fbdfab0 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:48.039832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65980418-77c8-485a-9ae0-7f361fbdfab0 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:48.044938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1506b1-ee73-4ef3-9627-987e65964a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:48.045622 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1506b1-ee73-4ef3-9627-987e65964a62 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:48.045978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1506b1-ee73-4ef3-9627-987e65964a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:48.046198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1506b1-ee73-4ef3-9627-987e65964a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:48.046842 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1506b1-ee73-4ef3-9627-987e65964a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:48.048980 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1506b1-ee73-4ef3-9627-987e65964a62 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:48.062369 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1346/2677] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:48.063605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332688c-e29c-4006-b796-b5aa6eaacf0c admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:48.074672 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332688c-e29c-4006-b796-b5aa6eaacf0c admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:48.079264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543aa046-9c51-4ae9-a5a9-805690fa2d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:48.079937 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543aa046-9c51-4ae9-a5a9-805690fa2d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:48.080208 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543aa046-9c51-4ae9-a5a9-805690fa2d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:48.080475 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543aa046-9c51-4ae9-a5a9-805690fa2d58 admin admin] PATH_INFO: `/v3/projects/430901616c38488299919df353382d11/users/c2f43a15f87f40fa9d4b1a9671cd4601/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:48.095593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543aa046-9c51-4ae9-a5a9-805690fa2d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=430901616c38488299919df353382d11, user_id=c2f43a15f87f40fa9d4b1a9671cd4601, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:48.098254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543aa046-9c51-4ae9-a5a9-805690fa2d58 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:48.112095 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1332/2678] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:48 2022] PUT /identity/v3/projects/430901616c38488299919df353382d11/users/c2f43a15f87f40fa9d4b1a9671cd4601/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:48.116357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1506b1-ee73-4ef3-9627-987e65964a62 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:48.127447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1506b1-ee73-4ef3-9627-987e65964a62 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:48.199215 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3ee6c-476d-49ee-b803-1a105b120d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:48.202147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3ee6c-476d-49ee-b803-1a105b120d97 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:48.202433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3ee6c-476d-49ee-b803-1a105b120d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:48.202717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3ee6c-476d-49ee-b803-1a105b120d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:48.203406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3ee6c-476d-49ee-b803-1a105b120d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:48.205974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3ee6c-476d-49ee-b803-1a105b120d97 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:48.220834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1347/2679] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:48.222409 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543aa046-9c51-4ae9-a5a9-805690fa2d58 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:48.233316 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543aa046-9c51-4ae9-a5a9-805690fa2d58 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:48.239077 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bd8e2-4f40-475a-a2f2-3d996d5ce000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:48.239944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bd8e2-4f40-475a-a2f2-3d996d5ce000 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:48.240158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bd8e2-4f40-475a-a2f2-3d996d5ce000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:48.240432 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bd8e2-4f40-475a-a2f2-3d996d5ce000 admin admin] PATH_INFO: `/v3/projects/430901616c38488299919df353382d11/users/c2f43a15f87f40fa9d4b1a9671cd4601/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:48.252331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bd8e2-4f40-475a-a2f2-3d996d5ce000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=430901616c38488299919df353382d11, user_id=c2f43a15f87f40fa9d4b1a9671cd4601, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:48.252331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bd8e2-4f40-475a-a2f2-3d996d5ce000 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:48.262156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1333/2680] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:48 2022] PUT /identity/v3/projects/430901616c38488299919df353382d11/users/c2f43a15f87f40fa9d4b1a9671cd4601/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:48.271956 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a556e-095c-437e-8ad4-b9156e6577f6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:48.272836 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a556e-095c-437e-8ad4-b9156e6577f6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:48.272836 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a556e-095c-437e-8ad4-b9156e6577f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:48.304720 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-782a556e-095c-437e-8ad4-b9156e6577f6 None None] MFA Rules not processed for user `c2f43a15f87f40fa9d4b1a9671cd4601`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:48.306203 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-782a556e-095c-437e-8ad4-b9156e6577f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:48.320736 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:48.384413 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1348/2681] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:48.396639 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bd8e2-4f40-475a-a2f2-3d996d5ce000 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:48.450530 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bd8e2-4f40-475a-a2f2-3d996d5ce000 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:48.454321 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4654bf-13d3-400b-9f77-6bc7efbcb322 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:48.455428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4654bf-13d3-400b-9f77-6bc7efbcb322 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:48.455704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4654bf-13d3-400b-9f77-6bc7efbcb322 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:48.455972 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4654bf-13d3-400b-9f77-6bc7efbcb322 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:48.464904 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4654bf-13d3-400b-9f77-6bc7efbcb322 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:48.466426 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4654bf-13d3-400b-9f77-6bc7efbcb322 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:48.476235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1334/2682] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:48.518465 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a556e-095c-437e-8ad4-b9156e6577f6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:48.519845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-782a556e-095c-437e-8ad4-b9156e6577f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:48.531573 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a556e-095c-437e-8ad4-b9156e6577f6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:48.535651 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afb518-8aac-4e89-b5bf-10bb5e19b0a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:48.536237 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afb518-8aac-4e89-b5bf-10bb5e19b0a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:48.536505 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afb518-8aac-4e89-b5bf-10bb5e19b0a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:48.536772 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afb518-8aac-4e89-b5bf-10bb5e19b0a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:48.593628 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afb518-8aac-4e89-b5bf-10bb5e19b0a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:48.595092 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afb518-8aac-4e89-b5bf-10bb5e19b0a3 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:48.604515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1349/2683] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:53.636765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4654bf-13d3-400b-9f77-6bc7efbcb322 service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:53.646047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4654bf-13d3-400b-9f77-6bc7efbcb322 service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:53.649812 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c64e2-da88-4721-8f73-3a97755070ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:53.650428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c64e2-da88-4721-8f73-3a97755070ce service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:53.650701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c64e2-da88-4721-8f73-3a97755070ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:53.650971 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c64e2-da88-4721-8f73-3a97755070ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:53.652715 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-db5c64e2-da88-4721-8f73-3a97755070ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:53.736304 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c64e2-da88-4721-8f73-3a97755070ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:53.737740 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c64e2-da88-4721-8f73-3a97755070ce service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:53.756681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1335/2684] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:54.671295 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afb518-8aac-4e89-b5bf-10bb5e19b0a3 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:54.748083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afb518-8aac-4e89-b5bf-10bb5e19b0a3 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:54.753496 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4bc2f-9a81-43a4-a723-d441c9b1c476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:54.754520 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4bc2f-9a81-43a4-a723-d441c9b1c476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:54.754801 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4bc2f-9a81-43a4-a723-d441c9b1c476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:54.755073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4bc2f-9a81-43a4-a723-d441c9b1c476 admin admin] PATH_INFO: `/v3/users/3a9794edf60e4190ade0ac7416f88bf7` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:54.756683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4bc2f-9a81-43a4-a723-d441c9b1c476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a9794edf60e4190ade0ac7416f88bf7)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:54.758703 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4bc2f-9a81-43a4-a723-d441c9b1c476 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:54.806010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:54.808181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e4e4bc2f-9a81-43a4-a723-d441c9b1c476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a9794edf60e4190ade0ac7416f88bf7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:54.808865 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e4e4bc2f-9a81-43a4-a723-d441c9b1c476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a9794edf60e4190ade0ac7416f88bf7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:54.818545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e4e4bc2f-9a81-43a4-a723-d441c9b1c476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a9794edf60e4190ade0ac7416f88bf7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:54.826964 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e4e4bc2f-9a81-43a4-a723-d441c9b1c476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a9794edf60e4190ade0ac7416f88bf7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:54.840853 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e4e4bc2f-9a81-43a4-a723-d441c9b1c476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a9794edf60e4190ade0ac7416f88bf7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:54.846854 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1350/2685] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:54 2022] DELETE /identity/v3/users/3a9794edf60e4190ade0ac7416f88bf7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:54.851358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52d605-475e-4a60-865b-288f750fc58b None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:54.851867 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52d605-475e-4a60-865b-288f750fc58b None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:54.852083 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52d605-475e-4a60-865b-288f750fc58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:54.870785 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-3f52d605-475e-4a60-865b-288f750fc58b None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:54.871974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-3f52d605-475e-4a60-865b-288f750fc58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:54.885420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:54.935061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1336/2686] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:54.945091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4bc2f-9a81-43a4-a723-d441c9b1c476 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:54.946512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e4e4bc2f-9a81-43a4-a723-d441c9b1c476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:55.008589 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4bc2f-9a81-43a4-a723-d441c9b1c476 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:55.012372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab6849-cf92-44e5-92e4-64d817bd27e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:55.012979 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab6849-cf92-44e5-92e4-64d817bd27e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:55.013256 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab6849-cf92-44e5-92e4-64d817bd27e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:55.013526 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab6849-cf92-44e5-92e4-64d817bd27e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:55.023182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab6849-cf92-44e5-92e4-64d817bd27e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:55.024596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab6849-cf92-44e5-92e4-64d817bd27e7 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:55.035004 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1351/2687] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:55.310420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52d605-475e-4a60-865b-288f750fc58b None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:55.311903 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-3f52d605-475e-4a60-865b-288f750fc58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:55.324553 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52d605-475e-4a60-865b-288f750fc58b None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:55.330825 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac782d-8d5f-4e77-84cc-e01a4416e661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:55.331762 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac782d-8d5f-4e77-84cc-e01a4416e661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:55.332399 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac782d-8d5f-4e77-84cc-e01a4416e661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:55.332399 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac782d-8d5f-4e77-84cc-e01a4416e661 admin admin] PATH_INFO: `/v3/projects/a1fdc67aa6b94d6f9772d3435cdda1a0` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:55.334231 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac782d-8d5f-4e77-84cc-e01a4416e661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1fdc67aa6b94d6f9772d3435cdda1a0)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:55.342704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac782d-8d5f-4e77-84cc-e01a4416e661 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:55.398200 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-12ac782d-8d5f-4e77-84cc-e01a4416e661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1fdc67aa6b94d6f9772d3435cdda1a0', 'request_id': 'req-12ac782d-8d5f-4e77-84cc-e01a4416e661'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:55.407061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-12ac782d-8d5f-4e77-84cc-e01a4416e661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1fdc67aa6b94d6f9772d3435cdda1a0', 'request_id': 'req-12ac782d-8d5f-4e77-84cc-e01a4416e661'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:55.413525 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-12ac782d-8d5f-4e77-84cc-e01a4416e661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1fdc67aa6b94d6f9772d3435cdda1a0', 'request_id': 'req-12ac782d-8d5f-4e77-84cc-e01a4416e661'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:55.424818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-12ac782d-8d5f-4e77-84cc-e01a4416e661 admin admin] The token cache is being invalidate because project a1fdc67aa6b94d6f9772d3435cdda1a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:55.425109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-12ac782d-8d5f-4e77-84cc-e01a4416e661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:55.425762 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-12ac782d-8d5f-4e77-84cc-e01a4416e661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:55.427284 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1337/2688] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:55 2022] DELETE /identity/v3/projects/a1fdc67aa6b94d6f9772d3435cdda1a0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:55.431275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab6849-cf92-44e5-92e4-64d817bd27e7 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:55.441560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab6849-cf92-44e5-92e4-64d817bd27e7 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:55.497297 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e3315-a494-42a2-9426-b7f3133951bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:55.497935 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e3315-a494-42a2-9426-b7f3133951bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:55.498253 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e3315-a494-42a2-9426-b7f3133951bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:55.498517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e3315-a494-42a2-9426-b7f3133951bd admin admin] PATH_INFO: `/v3/users/bf35d7be8f75409eb92d7d508f5316ac` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:55.499907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e3315-a494-42a2-9426-b7f3133951bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf35d7be8f75409eb92d7d508f5316ac)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:55.502083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e3315-a494-42a2-9426-b7f3133951bd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:55.550162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc6e3315-a494-42a2-9426-b7f3133951bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf35d7be8f75409eb92d7d508f5316ac'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:55.550824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc6e3315-a494-42a2-9426-b7f3133951bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf35d7be8f75409eb92d7d508f5316ac'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:55.557713 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc6e3315-a494-42a2-9426-b7f3133951bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf35d7be8f75409eb92d7d508f5316ac'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:55.565138 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc6e3315-a494-42a2-9426-b7f3133951bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf35d7be8f75409eb92d7d508f5316ac'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:55.569991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:55.581700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cc6e3315-a494-42a2-9426-b7f3133951bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf35d7be8f75409eb92d7d508f5316ac'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:55.587916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1352/2689] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:55 2022] DELETE /identity/v3/users/bf35d7be8f75409eb92d7d508f5316ac => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:55.863994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac782d-8d5f-4e77-84cc-e01a4416e661 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:55.865593 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-12ac782d-8d5f-4e77-84cc-e01a4416e661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:55.885131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac782d-8d5f-4e77-84cc-e01a4416e661 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:55.955020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a44ec-139d-43f3-8ecf-1b975d09e931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:55.955657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a44ec-139d-43f3-8ecf-1b975d09e931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:55.955938 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a44ec-139d-43f3-8ecf-1b975d09e931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:55.956209 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a44ec-139d-43f3-8ecf-1b975d09e931 admin admin] PATH_INFO: `/v3/projects/91579a604c914c6cb7c4df078ef8dcd6` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:55.957341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a44ec-139d-43f3-8ecf-1b975d09e931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91579a604c914c6cb7c4df078ef8dcd6)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:55.959435 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a44ec-139d-43f3-8ecf-1b975d09e931 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:56.004431 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:56.008338 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f65a44ec-139d-43f3-8ecf-1b975d09e931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91579a604c914c6cb7c4df078ef8dcd6', 'request_id': 'req-f65a44ec-139d-43f3-8ecf-1b975d09e931'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:56.017160 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f65a44ec-139d-43f3-8ecf-1b975d09e931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91579a604c914c6cb7c4df078ef8dcd6', 'request_id': 'req-f65a44ec-139d-43f3-8ecf-1b975d09e931'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:56.023649 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f65a44ec-139d-43f3-8ecf-1b975d09e931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91579a604c914c6cb7c4df078ef8dcd6', 'request_id': 'req-f65a44ec-139d-43f3-8ecf-1b975d09e931'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:56.036000 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f65a44ec-139d-43f3-8ecf-1b975d09e931 admin admin] The token cache is being invalidate because project 91579a604c914c6cb7c4df078ef8dcd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:56.036280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f65a44ec-139d-43f3-8ecf-1b975d09e931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:56.036960 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f65a44ec-139d-43f3-8ecf-1b975d09e931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:56.038560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1338/2690] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:55 2022] DELETE /identity/v3/projects/91579a604c914c6cb7c4df078ef8dcd6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:56.042885 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e3315-a494-42a2-9426-b7f3133951bd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:56.044513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cc6e3315-a494-42a2-9426-b7f3133951bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:56.059966 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e3315-a494-42a2-9426-b7f3133951bd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:56.120107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f1ab1-bc9a-4986-9b89-a394df0c9bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:56.120670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f1ab1-bc9a-4986-9b89-a394df0c9bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:56.120953 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f1ab1-bc9a-4986-9b89-a394df0c9bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:56.121228 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f1ab1-bc9a-4986-9b89-a394df0c9bb0 admin admin] PATH_INFO: `/v3/users/c2f43a15f87f40fa9d4b1a9671cd4601` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:56.122461 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f1ab1-bc9a-4986-9b89-a394df0c9bb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2f43a15f87f40fa9d4b1a9671cd4601)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:56.124408 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f1ab1-bc9a-4986-9b89-a394df0c9bb0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:56.184596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-327f1ab1-bc9a-4986-9b89-a394df0c9bb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2f43a15f87f40fa9d4b1a9671cd4601'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:56.185468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-327f1ab1-bc9a-4986-9b89-a394df0c9bb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2f43a15f87f40fa9d4b1a9671cd4601'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:56.196130 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-327f1ab1-bc9a-4986-9b89-a394df0c9bb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2f43a15f87f40fa9d4b1a9671cd4601'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:56.209531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-327f1ab1-bc9a-4986-9b89-a394df0c9bb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2f43a15f87f40fa9d4b1a9671cd4601'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:56.234052 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-327f1ab1-bc9a-4986-9b89-a394df0c9bb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2f43a15f87f40fa9d4b1a9671cd4601'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:56.243746 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1353/2691] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:56 2022] DELETE /identity/v3/users/c2f43a15f87f40fa9d4b1a9671cd4601 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:56.617845 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a44ec-139d-43f3-8ecf-1b975d09e931 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:56.619889 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f65a44ec-139d-43f3-8ecf-1b975d09e931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:56.642392 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a44ec-139d-43f3-8ecf-1b975d09e931 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:56.723137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0332b-6873-4f85-91f0-7a56579f2f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:56.723766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0332b-6873-4f85-91f0-7a56579f2f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:56.724042 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0332b-6873-4f85-91f0-7a56579f2f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:56.724309 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0332b-6873-4f85-91f0-7a56579f2f3d admin admin] PATH_INFO: `/v3/projects/430901616c38488299919df353382d11` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:56.725424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0332b-6873-4f85-91f0-7a56579f2f3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=430901616c38488299919df353382d11)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:56.727534 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0332b-6873-4f85-91f0-7a56579f2f3d admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:56.777082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-1db0332b-6873-4f85-91f0-7a56579f2f3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '430901616c38488299919df353382d11', 'request_id': 'req-1db0332b-6873-4f85-91f0-7a56579f2f3d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:56.784634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-1db0332b-6873-4f85-91f0-7a56579f2f3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '430901616c38488299919df353382d11', 'request_id': 'req-1db0332b-6873-4f85-91f0-7a56579f2f3d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:56.791519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-1db0332b-6873-4f85-91f0-7a56579f2f3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '430901616c38488299919df353382d11', 'request_id': 'req-1db0332b-6873-4f85-91f0-7a56579f2f3d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:56.803422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-1db0332b-6873-4f85-91f0-7a56579f2f3d admin admin] The token cache is being invalidate because project 430901616c38488299919df353382d11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:56.803963 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-1db0332b-6873-4f85-91f0-7a56579f2f3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:56.804796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-1db0332b-6873-4f85-91f0-7a56579f2f3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:56.806445 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1339/2692] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:56 2022] DELETE /identity/v3/projects/430901616c38488299919df353382d11 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:56.812348 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f5b32-e671-4f59-9f93-3f565c7d45ab None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:56.812628 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f5b32-e671-4f59-9f93-3f565c7d45ab None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:56.812894 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f5b32-e671-4f59-9f93-3f565c7d45ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:56.833485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-901f5b32-e671-4f59-9f93-3f565c7d45ab None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:56.834733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-901f5b32-e671-4f59-9f93-3f565c7d45ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:56.909340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1354/2693] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:56.911328 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616fc37-e413-4e48-9fde-7b2347d7ea23 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:56.911787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616fc37-e413-4e48-9fde-7b2347d7ea23 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:56.912074 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616fc37-e413-4e48-9fde-7b2347d7ea23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:56.935597 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-5616fc37-e413-4e48-9fde-7b2347d7ea23 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:56.937110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-5616fc37-e413-4e48-9fde-7b2347d7ea23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:56.946979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1340/2694] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:56.951431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f5b32-e671-4f59-9f93-3f565c7d45ab None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:56.960286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:56.963375 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f5b32-e671-4f59-9f93-3f565c7d45ab None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:56.976462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e769805-af2d-49bf-a5ba-329a3103453a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:56.977362 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e769805-af2d-49bf-a5ba-329a3103453a admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:56.981431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e769805-af2d-49bf-a5ba-329a3103453a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:56.981718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e769805-af2d-49bf-a5ba-329a3103453a admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:56.982678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e769805-af2d-49bf-a5ba-329a3103453a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:16:56.982995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e769805-af2d-49bf-a5ba-329a3103453a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:56.985213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e769805-af2d-49bf-a5ba-329a3103453a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:57.007986 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1355/2695] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:16:57.009962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616fc37-e413-4e48-9fde-7b2347d7ea23 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:57.020605 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616fc37-e413-4e48-9fde-7b2347d7ea23 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:57.026093 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a473e0-0c70-4ffa-b7a2-433dcdd8be21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:57.027046 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a473e0-0c70-4ffa-b7a2-433dcdd8be21 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:57.027481 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a473e0-0c70-4ffa-b7a2-433dcdd8be21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:57.027927 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a473e0-0c70-4ffa-b7a2-433dcdd8be21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:57.028878 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a473e0-0c70-4ffa-b7a2-433dcdd8be21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:57.031032 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a473e0-0c70-4ffa-b7a2-433dcdd8be21 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:57.065109 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1341/2696] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:57 2022] POST /identity/v3/projects => generated 361 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:57.069344 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e769805-af2d-49bf-a5ba-329a3103453a admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:57.080159 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e769805-af2d-49bf-a5ba-329a3103453a admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:57.148427 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa83f6-6802-4e33-99e0-0d2fce3af147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:57.149205 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa83f6-6802-4e33-99e0-0d2fce3af147 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:57.149548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa83f6-6802-4e33-99e0-0d2fce3af147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:57.149928 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa83f6-6802-4e33-99e0-0d2fce3af147 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:57.150719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa83f6-6802-4e33-99e0-0d2fce3af147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:57.153580 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa83f6-6802-4e33-99e0-0d2fce3af147 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:57.185918 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1356/2697] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:57 2022] POST /identity/v3/users => generated 281 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:57.187301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a473e0-0c70-4ffa-b7a2-433dcdd8be21 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:57.206398 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a473e0-0c70-4ffa-b7a2-433dcdd8be21 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:57.211995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee084cc-6a0b-43c9-a8d8-9619c38bd88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:57.212682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee084cc-6a0b-43c9-a8d8-9619c38bd88e admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:57.213347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee084cc-6a0b-43c9-a8d8-9619c38bd88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:57.213849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee084cc-6a0b-43c9-a8d8-9619c38bd88e admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:57.214536 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee084cc-6a0b-43c9-a8d8-9619c38bd88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:57.216252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee084cc-6a0b-43c9-a8d8-9619c38bd88e admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:57.232258 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1342/2698] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:57.236565 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa83f6-6802-4e33-99e0-0d2fce3af147 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:57.246984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa83f6-6802-4e33-99e0-0d2fce3af147 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:57.251587 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac27013-55d8-4523-a29d-2816aaa6c72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:57.252189 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac27013-55d8-4523-a29d-2816aaa6c72f admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:57.252459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac27013-55d8-4523-a29d-2816aaa6c72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:57.252726 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac27013-55d8-4523-a29d-2816aaa6c72f admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:57.253362 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac27013-55d8-4523-a29d-2816aaa6c72f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:57.254998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac27013-55d8-4523-a29d-2816aaa6c72f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:57.266045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1357/2699] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:57.270884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee084cc-6a0b-43c9-a8d8-9619c38bd88e admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:57.281082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee084cc-6a0b-43c9-a8d8-9619c38bd88e admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:57.287271 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a470f02-0775-4d92-a792-d219b4f19b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:57.288209 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a470f02-0775-4d92-a792-d219b4f19b9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:57.288401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a470f02-0775-4d92-a792-d219b4f19b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:57.288675 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a470f02-0775-4d92-a792-d219b4f19b9a admin admin] PATH_INFO: `/v3/projects/a70e1b15f3964166ae96941ce045e0ec/users/2f05e885ebf24e788de5deef90e06c5b/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:57.313708 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a470f02-0775-4d92-a792-d219b4f19b9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a70e1b15f3964166ae96941ce045e0ec, user_id=2f05e885ebf24e788de5deef90e06c5b, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:57.316629 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a470f02-0775-4d92-a792-d219b4f19b9a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:57.326440 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:57.334410 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1343/2700] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:57 2022] PUT /identity/v3/projects/a70e1b15f3964166ae96941ce045e0ec/users/2f05e885ebf24e788de5deef90e06c5b/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:57.350298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a703b50-a728-4600-992d-9cc506482c34 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:57.350578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a703b50-a728-4600-992d-9cc506482c34 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:57.350849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a703b50-a728-4600-992d-9cc506482c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:57.384077 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7a703b50-a728-4600-992d-9cc506482c34 None None] MFA Rules not processed for user `2f05e885ebf24e788de5deef90e06c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:57.385644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7a703b50-a728-4600-992d-9cc506482c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:57.477105 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1358/2701] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:57.503294 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a470f02-0775-4d92-a792-d219b4f19b9a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:57.507377 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0a470f02-0775-4d92-a792-d219b4f19b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:57.528523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a703b50-a728-4600-992d-9cc506482c34 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:57.530233 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7a703b50-a728-4600-992d-9cc506482c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:57.598909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a470f02-0775-4d92-a792-d219b4f19b9a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:57.602556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a703b50-a728-4600-992d-9cc506482c34 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:57.604184 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe20327-64e0-4c9d-a3d5-f2ce277bf5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:57.604974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe20327-64e0-4c9d-a3d5-f2ce277bf5a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:57.605167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe20327-64e0-4c9d-a3d5-f2ce277bf5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:57.605443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe20327-64e0-4c9d-a3d5-f2ce277bf5a5 admin admin] PATH_INFO: `/v3/users/fde5f401fed8452dadee6d5f581ad57f` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:57.606678 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe20327-64e0-4c9d-a3d5-f2ce277bf5a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde5f401fed8452dadee6d5f581ad57f)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:57.608142 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5e817-ffb9-4063-979f-90683cf3ff91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:57.608662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe20327-64e0-4c9d-a3d5-f2ce277bf5a5 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:57.608779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5e817-ffb9-4063-979f-90683cf3ff91 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:57.609023 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5e817-ffb9-4063-979f-90683cf3ff91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:57.609296 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5e817-ffb9-4063-979f-90683cf3ff91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:57.627698 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5e817-ffb9-4063-979f-90683cf3ff91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:57.630023 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5e817-ffb9-4063-979f-90683cf3ff91 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:57.644879 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1359/2702] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:16:57.663259 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5e817-ffb9-4063-979f-90683cf3ff91 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:57.676905 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:57.681897 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9fe20327-64e0-4c9d-a3d5-f2ce277bf5a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde5f401fed8452dadee6d5f581ad57f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:57.684263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5e817-ffb9-4063-979f-90683cf3ff91 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:57.691602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb24be-a467-46bb-b95f-ac74d4b3dd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:57.691910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb24be-a467-46bb-b95f-ac74d4b3dd85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:57.692159 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb24be-a467-46bb-b95f-ac74d4b3dd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:57.692456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb24be-a467-46bb-b95f-ac74d4b3dd85 admin admin] PATH_INFO: `/v3/users/2f05e885ebf24e788de5deef90e06c5b` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:57.693890 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb24be-a467-46bb-b95f-ac74d4b3dd85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f05e885ebf24e788de5deef90e06c5b)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:57.695389 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9fe20327-64e0-4c9d-a3d5-f2ce277bf5a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde5f401fed8452dadee6d5f581ad57f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:57.696022 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb24be-a467-46bb-b95f-ac74d4b3dd85 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:57.714767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9fe20327-64e0-4c9d-a3d5-f2ce277bf5a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde5f401fed8452dadee6d5f581ad57f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:57.732998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9fe20327-64e0-4c9d-a3d5-f2ce277bf5a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde5f401fed8452dadee6d5f581ad57f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:57.734156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-9fe20327-64e0-4c9d-a3d5-f2ce277bf5a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde5f401fed8452dadee6d5f581ad57f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:57.744342 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1344/2703] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:57 2022] DELETE /identity/v3/users/fde5f401fed8452dadee6d5f581ad57f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:57.749175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf6955-7d57-4816-b2ec-198a5ae0ae25 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:57.749464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf6955-7d57-4816-b2ec-198a5ae0ae25 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:57.749789 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf6955-7d57-4816-b2ec-198a5ae0ae25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:57.765117 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fdbb24be-a467-46bb-b95f-ac74d4b3dd85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f05e885ebf24e788de5deef90e06c5b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:57.766057 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fdbb24be-a467-46bb-b95f-ac74d4b3dd85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f05e885ebf24e788de5deef90e06c5b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:57.779279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-7ccf6955-7d57-4816-b2ec-198a5ae0ae25 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:57.780720 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fdbb24be-a467-46bb-b95f-ac74d4b3dd85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f05e885ebf24e788de5deef90e06c5b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:57.781741 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7ccf6955-7d57-4816-b2ec-198a5ae0ae25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:57.794617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fdbb24be-a467-46bb-b95f-ac74d4b3dd85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f05e885ebf24e788de5deef90e06c5b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:57.814540 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fdbb24be-a467-46bb-b95f-ac74d4b3dd85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f05e885ebf24e788de5deef90e06c5b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:57.822435 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1360/2704] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:57 2022] DELETE /identity/v3/users/2f05e885ebf24e788de5deef90e06c5b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:57.826959 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30ed98-4d3c-4e84-86b3-ca56625dce11 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:57.827612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30ed98-4d3c-4e84-86b3-ca56625dce11 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:57.828045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30ed98-4d3c-4e84-86b3-ca56625dce11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:57.852061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ad30ed98-4d3c-4e84-86b3-ca56625dce11 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:57.853444 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ad30ed98-4d3c-4e84-86b3-ca56625dce11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:57.875415 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1345/2705] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:57.888746 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf6955-7d57-4816-b2ec-198a5ae0ae25 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:57.891384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7ccf6955-7d57-4816-b2ec-198a5ae0ae25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:57.926452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:57.941530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1361/2706] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:57.953790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30ed98-4d3c-4e84-86b3-ca56625dce11 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:57.956139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ad30ed98-4d3c-4e84-86b3-ca56625dce11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:57.986939 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf6955-7d57-4816-b2ec-198a5ae0ae25 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:58.034649 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30ed98-4d3c-4e84-86b3-ca56625dce11 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:58.040357 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba58447-4b49-401f-bef9-e8f36f2497a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:58.040987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba58447-4b49-401f-bef9-e8f36f2497a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:58.041264 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba58447-4b49-401f-bef9-e8f36f2497a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:58.041554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba58447-4b49-401f-bef9-e8f36f2497a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:58.054789 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba58447-4b49-401f-bef9-e8f36f2497a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:58.056391 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba58447-4b49-401f-bef9-e8f36f2497a1 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:58.057140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda8b4c-6ad8-4047-87d0-317e86c17bad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:58.058059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda8b4c-6ad8-4047-87d0-317e86c17bad service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:58.058549 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda8b4c-6ad8-4047-87d0-317e86c17bad service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:58.059197 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda8b4c-6ad8-4047-87d0-317e86c17bad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:58.073349 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1362/2707] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:16:58.074682 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda8b4c-6ad8-4047-87d0-317e86c17bad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:58.076222 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda8b4c-6ad8-4047-87d0-317e86c17bad service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:58.090606 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1346/2708] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:16:58.733154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba58447-4b49-401f-bef9-e8f36f2497a1 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:58.734486 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6ba58447-4b49-401f-bef9-e8f36f2497a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:58.760344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda8b4c-6ad8-4047-87d0-317e86c17bad service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:58.770292 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-8bda8b4c-6ad8-4047-87d0-317e86c17bad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:58.770473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba58447-4b49-401f-bef9-e8f36f2497a1 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:58.779706 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda8b4c-6ad8-4047-87d0-317e86c17bad service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:58.785284 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:58.785284 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:58.785647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:58.785915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7 admin admin] PATH_INFO: `/v3/projects/47d2a1134608495d85e43e18995c8acc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:58.787355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47d2a1134608495d85e43e18995c8acc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:58.789543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:58.792736 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209b27b-2a9a-4c4a-ae50-ef0556d5bd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:58.795972 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209b27b-2a9a-4c4a-ae50-ef0556d5bd00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:58.796335 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209b27b-2a9a-4c4a-ae50-ef0556d5bd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:58.796691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209b27b-2a9a-4c4a-ae50-ef0556d5bd00 admin admin] PATH_INFO: `/v3/projects/a70e1b15f3964166ae96941ce045e0ec` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:58.799904 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209b27b-2a9a-4c4a-ae50-ef0556d5bd00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a70e1b15f3964166ae96941ce045e0ec)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:58.801901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209b27b-2a9a-4c4a-ae50-ef0556d5bd00 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:58.826815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:58.855583 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47d2a1134608495d85e43e18995c8acc', 'request_id': 'req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:58.864998 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47d2a1134608495d85e43e18995c8acc', 'request_id': 'req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:58.873833 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47d2a1134608495d85e43e18995c8acc', 'request_id': 'req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:58.889066 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4209b27b-2a9a-4c4a-ae50-ef0556d5bd00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a70e1b15f3964166ae96941ce045e0ec', 'request_id': 'req-4209b27b-2a9a-4c4a-ae50-ef0556d5bd00'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:58.892705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7 admin admin] The token cache is being invalidate because project 47d2a1134608495d85e43e18995c8acc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:58.893009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:58.894035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:58.896070 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1347/2709] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:58 2022] DELETE /identity/v3/projects/47d2a1134608495d85e43e18995c8acc => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:58.900706 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4209b27b-2a9a-4c4a-ae50-ef0556d5bd00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a70e1b15f3964166ae96941ce045e0ec', 'request_id': 'req-4209b27b-2a9a-4c4a-ae50-ef0556d5bd00'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:58.900942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:58.901875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:58.907203 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4209b27b-2a9a-4c4a-ae50-ef0556d5bd00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a70e1b15f3964166ae96941ce045e0ec', 'request_id': 'req-4209b27b-2a9a-4c4a-ae50-ef0556d5bd00'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:58.919824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4209b27b-2a9a-4c4a-ae50-ef0556d5bd00 admin admin] The token cache is being invalidate because project a70e1b15f3964166ae96941ce045e0ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:58.920116 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4209b27b-2a9a-4c4a-ae50-ef0556d5bd00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:58.920781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4209b27b-2a9a-4c4a-ae50-ef0556d5bd00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:58.922582 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1363/2710] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:58 2022] DELETE /identity/v3/projects/a70e1b15f3964166ae96941ce045e0ec => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:58.927310 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b8ff2-e5c9-4245-a00c-0a56a6d3d0cb None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:58.927592 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b8ff2-e5c9-4245-a00c-0a56a6d3d0cb None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:58.927866 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b8ff2-e5c9-4245-a00c-0a56a6d3d0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:58.947515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-cf2b8ff2-e5c9-4245-a00c-0a56a6d3d0cb None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:58.948725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cf2b8ff2-e5c9-4245-a00c-0a56a6d3d0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:58.982225 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf0743-f0ff-4e66-bce4-ca444e551bf7 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:58.986552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:59.011991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1364/2711] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:59.020227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64269d92-b2c2-4ded-88ed-65acaa9f2f74 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.020515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64269d92-b2c2-4ded-88ed-65acaa9f2f74 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.020790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64269d92-b2c2-4ded-88ed-65acaa9f2f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.041304 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-64269d92-b2c2-4ded-88ed-65acaa9f2f74 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:59.042573 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-64269d92-b2c2-4ded-88ed-65acaa9f2f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:59.050599 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bbb9b-7bd2-4553-a6c4-22d0e8f8b98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:59.051318 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bbb9b-7bd2-4553-a6c4-22d0e8f8b98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.051662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bbb9b-7bd2-4553-a6c4-22d0e8f8b98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.052001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bbb9b-7bd2-4553-a6c4-22d0e8f8b98a admin admin] PATH_INFO: `/v3/users/07ca58087fc242a6a8b8ea57d90b3cf9` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.053226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bbb9b-7bd2-4553-a6c4-22d0e8f8b98a admin admin] RBAC: Authorizing `identity:delete_user(user_id=07ca58087fc242a6a8b8ea57d90b3cf9)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:59.054779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1365/2712] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:59.055210 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bbb9b-7bd2-4553-a6c4-22d0e8f8b98a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:59.059612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64269d92-b2c2-4ded-88ed-65acaa9f2f74 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:59.070005 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64269d92-b2c2-4ded-88ed-65acaa9f2f74 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:59.076277 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0bdb1-8c4d-476f-9d46-46063d399a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:59.076955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0bdb1-8c4d-476f-9d46-46063d399a9b admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.077165 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0bdb1-8c4d-476f-9d46-46063d399a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.077442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0bdb1-8c4d-476f-9d46-46063d399a9b admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.078219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0bdb1-8c4d-476f-9d46-46063d399a9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:16:59.078524 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0bdb1-8c4d-476f-9d46-46063d399a9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:59.080775 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0bdb1-8c4d-476f-9d46-46063d399a9b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:59.094347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1366/2713] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:16:59.099673 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0bdb1-8c4d-476f-9d46-46063d399a9b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:59.111138 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0bdb1-8c4d-476f-9d46-46063d399a9b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:59.112842 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f68bbb9b-7bd2-4553-a6c4-22d0e8f8b98a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07ca58087fc242a6a8b8ea57d90b3cf9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:59.116852 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1be756-ba6b-447e-98e5-d06b8b1e74ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:59.118496 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1be756-ba6b-447e-98e5-d06b8b1e74ec admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.118496 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1be756-ba6b-447e-98e5-d06b8b1e74ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.118496 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1be756-ba6b-447e-98e5-d06b8b1e74ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.118777 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1be756-ba6b-447e-98e5-d06b8b1e74ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:59.121449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1be756-ba6b-447e-98e5-d06b8b1e74ec admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:59.123549 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f68bbb9b-7bd2-4553-a6c4-22d0e8f8b98a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07ca58087fc242a6a8b8ea57d90b3cf9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:59.137951 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f68bbb9b-7bd2-4553-a6c4-22d0e8f8b98a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07ca58087fc242a6a8b8ea57d90b3cf9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:59.147560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f68bbb9b-7bd2-4553-a6c4-22d0e8f8b98a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07ca58087fc242a6a8b8ea57d90b3cf9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:59.148312 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-f68bbb9b-7bd2-4553-a6c4-22d0e8f8b98a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07ca58087fc242a6a8b8ea57d90b3cf9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:59.155214 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1348/2714] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:58 2022] DELETE /identity/v3/users/07ca58087fc242a6a8b8ea57d90b3cf9 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:59.161433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1367/2715] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:59 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:59.161623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bbb9b-7bd2-4553-a6c4-22d0e8f8b98a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:59.161623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f68bbb9b-7bd2-4553-a6c4-22d0e8f8b98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:59.243483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bbb9b-7bd2-4553-a6c4-22d0e8f8b98a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:59.248783 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b93084-8ccc-4289-8c2a-b425c7fa0e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:59.249399 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b93084-8ccc-4289-8c2a-b425c7fa0e9c admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.249683 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b93084-8ccc-4289-8c2a-b425c7fa0e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.249956 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b93084-8ccc-4289-8c2a-b425c7fa0e9c admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.250659 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b93084-8ccc-4289-8c2a-b425c7fa0e9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:59.253161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b93084-8ccc-4289-8c2a-b425c7fa0e9c admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:59.281174 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1349/2716] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:59 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:59.282408 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1be756-ba6b-447e-98e5-d06b8b1e74ec admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:59.353719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1be756-ba6b-447e-98e5-d06b8b1e74ec admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:59.361362 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf485086-db3a-4dde-8244-497a94858288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:59.362093 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf485086-db3a-4dde-8244-497a94858288 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.362446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf485086-db3a-4dde-8244-497a94858288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.362725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf485086-db3a-4dde-8244-497a94858288 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.363429 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf485086-db3a-4dde-8244-497a94858288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:59.365112 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf485086-db3a-4dde-8244-497a94858288 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:59.366736 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:59.379227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1368/2717] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:59.383146 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b93084-8ccc-4289-8c2a-b425c7fa0e9c admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:59.394250 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b93084-8ccc-4289-8c2a-b425c7fa0e9c admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:59.399491 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfcd64-e87f-4991-88fd-4b4dda00c968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:59.400158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfcd64-e87f-4991-88fd-4b4dda00c968 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.400497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfcd64-e87f-4991-88fd-4b4dda00c968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.400834 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfcd64-e87f-4991-88fd-4b4dda00c968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.401546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfcd64-e87f-4991-88fd-4b4dda00c968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:59.403234 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfcd64-e87f-4991-88fd-4b4dda00c968 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:59.413936 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1350/2718] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:59.418260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf485086-db3a-4dde-8244-497a94858288 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:59.429161 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf485086-db3a-4dde-8244-497a94858288 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:59.434144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1259dc0-a89f-4e73-95d4-3e5c61238057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:59.434867 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1259dc0-a89f-4e73-95d4-3e5c61238057 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.435149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1259dc0-a89f-4e73-95d4-3e5c61238057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.435422 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1259dc0-a89f-4e73-95d4-3e5c61238057 admin admin] PATH_INFO: `/v3/projects/770c076806c64cd78d35f095ee489bcf/users/f849563115ec4a4e859a7395a03a561a/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.450720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfcd64-e87f-4991-88fd-4b4dda00c968 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:59.452992 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-cdbfcd64-e87f-4991-88fd-4b4dda00c968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:59.455997 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1259dc0-a89f-4e73-95d4-3e5c61238057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=770c076806c64cd78d35f095ee489bcf, user_id=f849563115ec4a4e859a7395a03a561a, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:59.458721 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1259dc0-a89f-4e73-95d4-3e5c61238057 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:59.465192 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:59.470403 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfcd64-e87f-4991-88fd-4b4dda00c968 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:59.478687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1369/2719] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:59 2022] PUT /identity/v3/projects/770c076806c64cd78d35f095ee489bcf/users/f849563115ec4a4e859a7395a03a561a/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:59.484022 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee458cf9-1ec8-48a1-95fc-ec670d606b66 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.484340 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee458cf9-1ec8-48a1-95fc-ec670d606b66 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.484684 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee458cf9-1ec8-48a1-95fc-ec670d606b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.505911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ee458cf9-1ec8-48a1-95fc-ec670d606b66 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:59.508213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ee458cf9-1ec8-48a1-95fc-ec670d606b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:59.542597 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd92533-f120-4272-ba8b-c64fd36346c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:59.543153 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd92533-f120-4272-ba8b-c64fd36346c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.543435 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd92533-f120-4272-ba8b-c64fd36346c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.543705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd92533-f120-4272-ba8b-c64fd36346c0 admin admin] PATH_INFO: `/v3/projects/6f09a326bf644d918828a5b9a554c5c5` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.544849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd92533-f120-4272-ba8b-c64fd36346c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f09a326bf644d918828a5b9a554c5c5)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:59.546824 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd92533-f120-4272-ba8b-c64fd36346c0 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:59.568974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1370/2720] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:59.579158 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee458cf9-1ec8-48a1-95fc-ec670d606b66 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:59.580962 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ee458cf9-1ec8-48a1-95fc-ec670d606b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:59.601286 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fcd92533-f120-4272-ba8b-c64fd36346c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f09a326bf644d918828a5b9a554c5c5', 'request_id': 'req-fcd92533-f120-4272-ba8b-c64fd36346c0'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:59.609589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fcd92533-f120-4272-ba8b-c64fd36346c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f09a326bf644d918828a5b9a554c5c5', 'request_id': 'req-fcd92533-f120-4272-ba8b-c64fd36346c0'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:59.616359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fcd92533-f120-4272-ba8b-c64fd36346c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f09a326bf644d918828a5b9a554c5c5', 'request_id': 'req-fcd92533-f120-4272-ba8b-c64fd36346c0'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:59.629128 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fcd92533-f120-4272-ba8b-c64fd36346c0 admin admin] The token cache is being invalidate because project 6f09a326bf644d918828a5b9a554c5c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:59.629416 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fcd92533-f120-4272-ba8b-c64fd36346c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:59.630149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fcd92533-f120-4272-ba8b-c64fd36346c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:59.631674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1351/2721] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:59 2022] DELETE /identity/v3/projects/6f09a326bf644d918828a5b9a554c5c5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:59.636301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ad86b-d448-479f-a44c-eff3910e221d None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.636585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ad86b-d448-479f-a44c-eff3910e221d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.636864 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ad86b-d448-479f-a44c-eff3910e221d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.644211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee458cf9-1ec8-48a1-95fc-ec670d606b66 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:59.655939 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-a12ad86b-d448-479f-a44c-eff3910e221d None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:59.657145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-a12ad86b-d448-479f-a44c-eff3910e221d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:59.700725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a730b-3887-4a7f-907a-d80ebd90143e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:59.701542 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a730b-3887-4a7f-907a-d80ebd90143e service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.701830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a730b-3887-4a7f-907a-d80ebd90143e service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.702142 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a730b-3887-4a7f-907a-d80ebd90143e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.704131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-db4a730b-3887-4a7f-907a-d80ebd90143e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:59.722600 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1352/2722] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:59.732164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05d39a-1068-4f08-949c-928b0b2034d3 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.732561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05d39a-1068-4f08-949c-928b0b2034d3 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.733092 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05d39a-1068-4f08-949c-928b0b2034d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.749885 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:59.754276 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-6f05d39a-1068-4f08-949c-928b0b2034d3 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:59.755538 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6f05d39a-1068-4f08-949c-928b0b2034d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:59.763514 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1353/2723] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:59.767696 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05d39a-1068-4f08-949c-928b0b2034d3 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:59.773137 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a730b-3887-4a7f-907a-d80ebd90143e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:59.774613 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a730b-3887-4a7f-907a-d80ebd90143e service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:59.776773 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05d39a-1068-4f08-949c-928b0b2034d3 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:59.781379 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b7ae1-e0c5-4455-9d10-b998af948f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:59.781962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b7ae1-e0c5-4455-9d10-b998af948f47 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.782272 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b7ae1-e0c5-4455-9d10-b998af948f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.782583 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b7ae1-e0c5-4455-9d10-b998af948f47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.783221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b7ae1-e0c5-4455-9d10-b998af948f47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:16:59.783513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b7ae1-e0c5-4455-9d10-b998af948f47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:59.784815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1371/2724] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:16:59.785110 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b7ae1-e0c5-4455-9d10-b998af948f47 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:59.800763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1354/2725] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:16:59.801974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a730b-3887-4a7f-907a-d80ebd90143e service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:59.812285 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a730b-3887-4a7f-907a-d80ebd90143e service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:59.817512 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68f59b-7878-47cd-b44d-799a4bfbecb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:59.818167 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68f59b-7878-47cd-b44d-799a4bfbecb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.818441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68f59b-7878-47cd-b44d-799a4bfbecb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.819168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68f59b-7878-47cd-b44d-799a4bfbecb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.819314 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68f59b-7878-47cd-b44d-799a4bfbecb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:59.821227 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68f59b-7878-47cd-b44d-799a4bfbecb0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:59.854208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1372/2726] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:59 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:59.856218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b7ae1-e0c5-4455-9d10-b998af948f47 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:59.868299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b7ae1-e0c5-4455-9d10-b998af948f47 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:59.883781 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:59.937254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f999e-8e99-4312-9e1f-9eb4a7f319c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:59.937644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f999e-8e99-4312-9e1f-9eb4a7f319c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.937925 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f999e-8e99-4312-9e1f-9eb4a7f319c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.938290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f999e-8e99-4312-9e1f-9eb4a7f319c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:59.939599 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f999e-8e99-4312-9e1f-9eb4a7f319c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:59.945408 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f999e-8e99-4312-9e1f-9eb4a7f319c3 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:59.976812 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1355/2727] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:59 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:16:59.977413 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68f59b-7878-47cd-b44d-799a4bfbecb0 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:59.992317 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68f59b-7878-47cd-b44d-799a4bfbecb0 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:59.997695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b1a6d-5478-42d2-a842-52af4e40ec56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:59.998462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b1a6d-5478-42d2-a842-52af4e40ec56 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:59.998908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b1a6d-5478-42d2-a842-52af4e40ec56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:59.999338 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b1a6d-5478-42d2-a842-52af4e40ec56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:00.000056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b1a6d-5478-42d2-a842-52af4e40ec56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:00.003855 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b1a6d-5478-42d2-a842-52af4e40ec56 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:00.022162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1373/2728] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:17:00.027598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f999e-8e99-4312-9e1f-9eb4a7f319c3 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:00.040559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f999e-8e99-4312-9e1f-9eb4a7f319c3 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:00.047198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa65fc-d1b1-4662-91a5-99e13494c1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:00.048478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa65fc-d1b1-4662-91a5-99e13494c1ae admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:00.048478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa65fc-d1b1-4662-91a5-99e13494c1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:00.048478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa65fc-d1b1-4662-91a5-99e13494c1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:00.049744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa65fc-d1b1-4662-91a5-99e13494c1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:00.051774 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa65fc-d1b1-4662-91a5-99e13494c1ae admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:00.065465 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1356/2729] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:17:00.069487 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b1a6d-5478-42d2-a842-52af4e40ec56 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:00.081248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b1a6d-5478-42d2-a842-52af4e40ec56 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:00.089314 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0989939-5d2e-41e7-b505-9ced0569983e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:00.090567 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0989939-5d2e-41e7-b505-9ced0569983e admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:00.090802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0989939-5d2e-41e7-b505-9ced0569983e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:00.091168 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0989939-5d2e-41e7-b505-9ced0569983e admin admin] PATH_INFO: `/v3/projects/8439254256a04b3f881bc7b70e97f172/users/4a4696186b874ed69235fbab1ffa592f/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:00.108290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0989939-5d2e-41e7-b505-9ced0569983e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8439254256a04b3f881bc7b70e97f172, user_id=4a4696186b874ed69235fbab1ffa592f, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:00.113684 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0989939-5d2e-41e7-b505-9ced0569983e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:00.133273 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1374/2730] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:17:00 2022] PUT /identity/v3/projects/8439254256a04b3f881bc7b70e97f172/users/4a4696186b874ed69235fbab1ffa592f/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:00.145331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400b7bc-eaee-4598-a6eb-b2acaaa43e10 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:00.145615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400b7bc-eaee-4598-a6eb-b2acaaa43e10 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:00.145883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400b7bc-eaee-4598-a6eb-b2acaaa43e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:00.187674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-0400b7bc-eaee-4598-a6eb-b2acaaa43e10 None None] MFA Rules not processed for user `4a4696186b874ed69235fbab1ffa592f`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:00.189883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0400b7bc-eaee-4598-a6eb-b2acaaa43e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:00.271939 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1357/2731] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:17:00.276314 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0989939-5d2e-41e7-b505-9ced0569983e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:00.354690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0989939-5d2e-41e7-b505-9ced0569983e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:00.359347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46c1cf-dc92-42f8-9cee-d2d47a9aa778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:00.359977 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46c1cf-dc92-42f8-9cee-d2d47a9aa778 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:00.360251 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46c1cf-dc92-42f8-9cee-d2d47a9aa778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:00.360519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46c1cf-dc92-42f8-9cee-d2d47a9aa778 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:00.362008 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46c1cf-dc92-42f8-9cee-d2d47a9aa778 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:00.363928 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46c1cf-dc92-42f8-9cee-d2d47a9aa778 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:00.397791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1375/2732] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:17:00 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:17:00.399206 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400b7bc-eaee-4598-a6eb-b2acaaa43e10 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:00.411559 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400b7bc-eaee-4598-a6eb-b2acaaa43e10 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:00.482183 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7433ec7-ccf5-48df-8ebc-3c0f19abe438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:00.482975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7433ec7-ccf5-48df-8ebc-3c0f19abe438 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:00.483230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7433ec7-ccf5-48df-8ebc-3c0f19abe438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:00.483455 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7433ec7-ccf5-48df-8ebc-3c0f19abe438 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:00.484080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7433ec7-ccf5-48df-8ebc-3c0f19abe438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:00.486633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7433ec7-ccf5-48df-8ebc-3c0f19abe438 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:00.522502 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1358/2733] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:17:00 2022] POST /identity/v3/users => generated 298 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:17:00.523753 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c42a80-7db3-4377-89a1-ba839dc614d6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:00.523910 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7433ec7-ccf5-48df-8ebc-3c0f19abe438 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:00.524221 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c42a80-7db3-4377-89a1-ba839dc614d6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:00.524500 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c42a80-7db3-4377-89a1-ba839dc614d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:00.535208 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7433ec7-ccf5-48df-8ebc-3c0f19abe438 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:00.540286 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15547e2e-9aeb-438e-96bf-cb7225318d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:00.541143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15547e2e-9aeb-438e-96bf-cb7225318d1f admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:00.541520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15547e2e-9aeb-438e-96bf-cb7225318d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:00.541898 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15547e2e-9aeb-438e-96bf-cb7225318d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:00.542709 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15547e2e-9aeb-438e-96bf-cb7225318d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:00.544396 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15547e2e-9aeb-438e-96bf-cb7225318d1f admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:00.556687 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1359/2734] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:17:00.560268 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15547e2e-9aeb-438e-96bf-cb7225318d1f admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:00.562841 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-65c42a80-7db3-4377-89a1-ba839dc614d6 None None] MFA Rules not processed for user `f849563115ec4a4e859a7395a03a561a`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:00.564893 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-65c42a80-7db3-4377-89a1-ba839dc614d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:00.566371 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:00.574455 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15547e2e-9aeb-438e-96bf-cb7225318d1f admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:00.582831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f699e-31b0-46c7-82c8-c3e03edf86c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:00.583650 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f699e-31b0-46c7-82c8-c3e03edf86c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:00.584048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f699e-31b0-46c7-82c8-c3e03edf86c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:00.584422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f699e-31b0-46c7-82c8-c3e03edf86c0 admin admin] PATH_INFO: `/v3/projects/31ccba6ee98d4bd2a53ef65dac2cf0b2/users/6cc6babdf51a458a939a5dbdef9878f3/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:00.602244 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f699e-31b0-46c7-82c8-c3e03edf86c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ccba6ee98d4bd2a53ef65dac2cf0b2, user_id=6cc6babdf51a458a939a5dbdef9878f3, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:00.605928 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f699e-31b0-46c7-82c8-c3e03edf86c0 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:00.622844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1360/2735] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:17:00 2022] PUT /identity/v3/projects/31ccba6ee98d4bd2a53ef65dac2cf0b2/users/6cc6babdf51a458a939a5dbdef9878f3/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:17:00.627580 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f699e-31b0-46c7-82c8-c3e03edf86c0 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:00.638013 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:00.649175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1376/2736] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:00.660541 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c42a80-7db3-4377-89a1-ba839dc614d6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:00.662208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-65c42a80-7db3-4377-89a1-ba839dc614d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:00.701615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f699e-31b0-46c7-82c8-c3e03edf86c0 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:00.707657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01accf-9647-4a35-a199-ac7e4e819793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:00.708134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01accf-9647-4a35-a199-ac7e4e819793 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:00.708381 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01accf-9647-4a35-a199-ac7e4e819793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:00.708651 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01accf-9647-4a35-a199-ac7e4e819793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:00.709537 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01accf-9647-4a35-a199-ac7e4e819793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:00.711252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01accf-9647-4a35-a199-ac7e4e819793 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:00.725186 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1361/2737] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:17:00.726340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01accf-9647-4a35-a199-ac7e4e819793 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:00.734159 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c42a80-7db3-4377-89a1-ba839dc614d6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:00.738721 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01accf-9647-4a35-a199-ac7e4e819793 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:00.743572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ffb60-c64c-4d5e-b89d-a21d459ee9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:00.744558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ffb60-c64c-4d5e-b89d-a21d459ee9f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:00.744911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ffb60-c64c-4d5e-b89d-a21d459ee9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:00.745254 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ffb60-c64c-4d5e-b89d-a21d459ee9f4 admin admin] PATH_INFO: `/v3/projects/31ccba6ee98d4bd2a53ef65dac2cf0b2/users/6cc6babdf51a458a939a5dbdef9878f3/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:00.747312 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ffb60-c64c-4d5e-b89d-a21d459ee9f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ccba6ee98d4bd2a53ef65dac2cf0b2, user_id=6cc6babdf51a458a939a5dbdef9878f3, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:00.750073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ffb60-c64c-4d5e-b89d-a21d459ee9f4 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:00.761589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1362/2738] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:17:00 2022] PUT /identity/v3/projects/31ccba6ee98d4bd2a53ef65dac2cf0b2/users/6cc6babdf51a458a939a5dbdef9878f3/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:17:00.772451 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798213d-d93b-4700-b3c7-e038c2459564 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:00.772734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798213d-d93b-4700-b3c7-e038c2459564 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:00.773012 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798213d-d93b-4700-b3c7-e038c2459564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:00.790635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41291c-aac5-49ac-b4b4-da9ad1043cfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:00.791240 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41291c-aac5-49ac-b4b4-da9ad1043cfa service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:00.791516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41291c-aac5-49ac-b4b4-da9ad1043cfa service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:00.791813 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41291c-aac5-49ac-b4b4-da9ad1043cfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:00.801551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-2798213d-d93b-4700-b3c7-e038c2459564 None None] MFA Rules not processed for user `6cc6babdf51a458a939a5dbdef9878f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:00.802823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2798213d-d93b-4700-b3c7-e038c2459564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:00.854154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41291c-aac5-49ac-b4b4-da9ad1043cfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:00.855614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41291c-aac5-49ac-b4b4-da9ad1043cfa service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:00.864871 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1377/2739] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:17:00.871030 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1363/2740] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:17:00.883063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41291c-aac5-49ac-b4b4-da9ad1043cfa service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:00.884623 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-dd41291c-aac5-49ac-b4b4-da9ad1043cfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:00.896268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41291c-aac5-49ac-b4b4-da9ad1043cfa service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:00.946530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4345f-13e7-4e26-b3fe-0245c5c069a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:00.947145 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4345f-13e7-4e26-b3fe-0245c5c069a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:00.947431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4345f-13e7-4e26-b3fe-0245c5c069a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:00.947808 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4345f-13e7-4e26-b3fe-0245c5c069a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:00.957589 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4345f-13e7-4e26-b3fe-0245c5c069a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:00.959083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4345f-13e7-4e26-b3fe-0245c5c069a3 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:00.968658 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1378/2741] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:17:01.044358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798213d-d93b-4700-b3c7-e038c2459564 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:01.054159 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798213d-d93b-4700-b3c7-e038c2459564 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:01.057734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5f4d3-464b-42f6-93d7-7ae358100f84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:01.058378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5f4d3-464b-42f6-93d7-7ae358100f84 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:01.058647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5f4d3-464b-42f6-93d7-7ae358100f84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:01.058909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5f4d3-464b-42f6-93d7-7ae358100f84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:01.114560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:01.134145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5f4d3-464b-42f6-93d7-7ae358100f84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:01.134145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5f4d3-464b-42f6-93d7-7ae358100f84 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:01.140865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1364/2742] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:17:01.319178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4345f-13e7-4e26-b3fe-0245c5c069a3 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:01.321106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-87e4345f-13e7-4e26-b3fe-0245c5c069a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:01.333065 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:01.387454 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4345f-13e7-4e26-b3fe-0245c5c069a3 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:01.392622 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559acb70-200f-4017-ab30-3e0cd6ba38d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:01.393252 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559acb70-200f-4017-ab30-3e0cd6ba38d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:01.393556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559acb70-200f-4017-ab30-3e0cd6ba38d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:01.393800 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559acb70-200f-4017-ab30-3e0cd6ba38d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:01.404796 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-559acb70-200f-4017-ab30-3e0cd6ba38d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:01.461900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5f4d3-464b-42f6-93d7-7ae358100f84 service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:01.477797 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559acb70-200f-4017-ab30-3e0cd6ba38d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:01.479376 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559acb70-200f-4017-ab30-3e0cd6ba38d8 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:01.489476 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1379/2743] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:17:01.526942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5f4d3-464b-42f6-93d7-7ae358100f84 service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:01.531526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d5c91-b68c-4f46-9f0f-fc8285f80b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:01.532158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d5c91-b68c-4f46-9f0f-fc8285f80b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:01.532462 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d5c91-b68c-4f46-9f0f-fc8285f80b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:01.532739 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d5c91-b68c-4f46-9f0f-fc8285f80b85 admin admin] PATH_INFO: `/v3/users/4a4696186b874ed69235fbab1ffa592f` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:01.533895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d5c91-b68c-4f46-9f0f-fc8285f80b85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a4696186b874ed69235fbab1ffa592f)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:01.535884 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d5c91-b68c-4f46-9f0f-fc8285f80b85 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:01.588607 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-522d5c91-b68c-4f46-9f0f-fc8285f80b85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a4696186b874ed69235fbab1ffa592f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:01.596164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-522d5c91-b68c-4f46-9f0f-fc8285f80b85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a4696186b874ed69235fbab1ffa592f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:01.608832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-522d5c91-b68c-4f46-9f0f-fc8285f80b85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a4696186b874ed69235fbab1ffa592f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:01.616052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-522d5c91-b68c-4f46-9f0f-fc8285f80b85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a4696186b874ed69235fbab1ffa592f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:01.616761 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-522d5c91-b68c-4f46-9f0f-fc8285f80b85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a4696186b874ed69235fbab1ffa592f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:01.623035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1365/2744] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:17:01 2022] DELETE /identity/v3/users/4a4696186b874ed69235fbab1ffa592f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:17:01.628014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e545d-f098-4fa0-919f-3f035d460eee None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:01.628444 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e545d-f098-4fa0-919f-3f035d460eee None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:01.628781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e545d-f098-4fa0-919f-3f035d460eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:01.649198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d08e545d-f098-4fa0-919f-3f035d460eee None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:01.650456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d08e545d-f098-4fa0-919f-3f035d460eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:01.713814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1380/2745] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:17:01.723996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d5c91-b68c-4f46-9f0f-fc8285f80b85 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:01.725439 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-522d5c91-b68c-4f46-9f0f-fc8285f80b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:01.789658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d5c91-b68c-4f46-9f0f-fc8285f80b85 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:01.793455 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6434a-69f8-40d1-952a-67f3f85082a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:01.794061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6434a-69f8-40d1-952a-67f3f85082a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:01.794402 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6434a-69f8-40d1-952a-67f3f85082a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:01.794673 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6434a-69f8-40d1-952a-67f3f85082a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:01.804825 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6434a-69f8-40d1-952a-67f3f85082a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:01.806301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6434a-69f8-40d1-952a-67f3f85082a7 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:01.817268 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1366/2746] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:17:02.103266 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e545d-f098-4fa0-919f-3f035d460eee None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:02.104731 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d08e545d-f098-4fa0-919f-3f035d460eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:02.118041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e545d-f098-4fa0-919f-3f035d460eee None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:02.124597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267dea8-c702-4ed6-995f-7b7c784f0113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:02.125516 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267dea8-c702-4ed6-995f-7b7c784f0113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:02.125956 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267dea8-c702-4ed6-995f-7b7c784f0113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:02.126583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267dea8-c702-4ed6-995f-7b7c784f0113 admin admin] PATH_INFO: `/v3/projects/8439254256a04b3f881bc7b70e97f172` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:02.128009 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8267dea8-c702-4ed6-995f-7b7c784f0113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8439254256a04b3f881bc7b70e97f172)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:02.130256 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8267dea8-c702-4ed6-995f-7b7c784f0113 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:02.185687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8267dea8-c702-4ed6-995f-7b7c784f0113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8439254256a04b3f881bc7b70e97f172', 'request_id': 'req-8267dea8-c702-4ed6-995f-7b7c784f0113'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:02.190081 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:02.202481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8267dea8-c702-4ed6-995f-7b7c784f0113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8439254256a04b3f881bc7b70e97f172', 'request_id': 'req-8267dea8-c702-4ed6-995f-7b7c784f0113'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:02.213095 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8267dea8-c702-4ed6-995f-7b7c784f0113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8439254256a04b3f881bc7b70e97f172', 'request_id': 'req-8267dea8-c702-4ed6-995f-7b7c784f0113'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:02.237313 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8267dea8-c702-4ed6-995f-7b7c784f0113 admin admin] The token cache is being invalidate because project 8439254256a04b3f881bc7b70e97f172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:17:02.238058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8267dea8-c702-4ed6-995f-7b7c784f0113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:02.238468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8267dea8-c702-4ed6-995f-7b7c784f0113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:02.243791 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1381/2747] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:17:02 2022] DELETE /identity/v3/projects/8439254256a04b3f881bc7b70e97f172 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:02.248055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6434a-69f8-40d1-952a-67f3f85082a7 service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:02.268997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6434a-69f8-40d1-952a-67f3f85082a7 service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:02.275494 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:02.340405 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05488c3-1ff8-4df3-b45b-168f53b1ed46 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:02.341104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05488c3-1ff8-4df3-b45b-168f53b1ed46 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:02.341104 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05488c3-1ff8-4df3-b45b-168f53b1ed46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:02.349277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa0b89-a845-4497-81e7-4d8b2ab6d649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:02.350765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa0b89-a845-4497-81e7-4d8b2ab6d649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:02.351059 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa0b89-a845-4497-81e7-4d8b2ab6d649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:02.351336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa0b89-a845-4497-81e7-4d8b2ab6d649 admin admin] PATH_INFO: `/v3/users/6cc6babdf51a458a939a5dbdef9878f3` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:02.352667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa0b89-a845-4497-81e7-4d8b2ab6d649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cc6babdf51a458a939a5dbdef9878f3)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:02.354638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa0b89-a845-4497-81e7-4d8b2ab6d649 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:02.362330 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d05488c3-1ff8-4df3-b45b-168f53b1ed46 None None] MFA Rules not processed for user `f849563115ec4a4e859a7395a03a561a`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:02.363453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d05488c3-1ff8-4df3-b45b-168f53b1ed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:02.413193 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-67fa0b89-a845-4497-81e7-4d8b2ab6d649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cc6babdf51a458a939a5dbdef9878f3'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:02.422288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-67fa0b89-a845-4497-81e7-4d8b2ab6d649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cc6babdf51a458a939a5dbdef9878f3'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:02.440427 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-67fa0b89-a845-4497-81e7-4d8b2ab6d649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cc6babdf51a458a939a5dbdef9878f3'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:02.448299 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1382/2748] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:17:02.457131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-67fa0b89-a845-4497-81e7-4d8b2ab6d649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cc6babdf51a458a939a5dbdef9878f3'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:02.458040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-67fa0b89-a845-4497-81e7-4d8b2ab6d649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cc6babdf51a458a939a5dbdef9878f3'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:02.464504 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1367/2749] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:17:02 2022] DELETE /identity/v3/users/6cc6babdf51a458a939a5dbdef9878f3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:17:03.040559 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05488c3-1ff8-4df3-b45b-168f53b1ed46 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:03.042488 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d05488c3-1ff8-4df3-b45b-168f53b1ed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:03.073421 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05488c3-1ff8-4df3-b45b-168f53b1ed46 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:03.164587 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6ea19-5041-420a-a551-02c2fb6c8c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:03.165617 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6ea19-5041-420a-a551-02c2fb6c8c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:03.166049 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6ea19-5041-420a-a551-02c2fb6c8c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:03.166496 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6ea19-5041-420a-a551-02c2fb6c8c0d admin admin] PATH_INFO: `/v3/projects/31ccba6ee98d4bd2a53ef65dac2cf0b2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:03.169594 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6ea19-5041-420a-a551-02c2fb6c8c0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=31ccba6ee98d4bd2a53ef65dac2cf0b2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:03.172097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6ea19-5041-420a-a551-02c2fb6c8c0d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:03.270310 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-faa6ea19-5041-420a-a551-02c2fb6c8c0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31ccba6ee98d4bd2a53ef65dac2cf0b2', 'request_id': 'req-faa6ea19-5041-420a-a551-02c2fb6c8c0d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:03.282844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-faa6ea19-5041-420a-a551-02c2fb6c8c0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31ccba6ee98d4bd2a53ef65dac2cf0b2', 'request_id': 'req-faa6ea19-5041-420a-a551-02c2fb6c8c0d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:03.290030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-faa6ea19-5041-420a-a551-02c2fb6c8c0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31ccba6ee98d4bd2a53ef65dac2cf0b2', 'request_id': 'req-faa6ea19-5041-420a-a551-02c2fb6c8c0d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:03.308678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-faa6ea19-5041-420a-a551-02c2fb6c8c0d admin admin] The token cache is being invalidate because project 31ccba6ee98d4bd2a53ef65dac2cf0b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:17:03.309031 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-faa6ea19-5041-420a-a551-02c2fb6c8c0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:03.309750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-faa6ea19-5041-420a-a551-02c2fb6c8c0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:03.315759 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1383/2750] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:17:03 2022] DELETE /identity/v3/projects/31ccba6ee98d4bd2a53ef65dac2cf0b2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:03.317395 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9395d6-4fb5-46df-ac4d-f603de3f351a None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:03.317890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9395d6-4fb5-46df-ac4d-f603de3f351a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:03.318046 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9395d6-4fb5-46df-ac4d-f603de3f351a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:03.339746 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-6f9395d6-4fb5-46df-ac4d-f603de3f351a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:03.340996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6f9395d6-4fb5-46df-ac4d-f603de3f351a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:03.419313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1368/2751] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:03.433412 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a860038-7038-4777-afdc-68c1585d09af None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:03.434485 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a860038-7038-4777-afdc-68c1585d09af None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:03.434940 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a860038-7038-4777-afdc-68c1585d09af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:03.463863 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1a860038-7038-4777-afdc-68c1585d09af None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:03.466056 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1a860038-7038-4777-afdc-68c1585d09af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:03.483668 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1384/2752] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:03.487748 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9395d6-4fb5-46df-ac4d-f603de3f351a None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:03.498804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9395d6-4fb5-46df-ac4d-f603de3f351a None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:03.505743 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a572a7-0836-4527-afd2-542d8a5c09d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:03.506558 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a572a7-0836-4527-afd2-542d8a5c09d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:03.506917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a572a7-0836-4527-afd2-542d8a5c09d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:03.508892 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a572a7-0836-4527-afd2-542d8a5c09d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:03.509665 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a572a7-0836-4527-afd2-542d8a5c09d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:17:03.510046 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a572a7-0836-4527-afd2-542d8a5c09d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:03.511811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a572a7-0836-4527-afd2-542d8a5c09d6 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:03.526403 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1369/2753] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:17:03.531132 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a860038-7038-4777-afdc-68c1585d09af None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:03.545224 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a860038-7038-4777-afdc-68c1585d09af None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:03.551204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b10a3-83a5-47d6-aa28-2f9fe56be8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:03.551947 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b10a3-83a5-47d6-aa28-2f9fe56be8aa admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:03.552310 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b10a3-83a5-47d6-aa28-2f9fe56be8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:03.552645 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b10a3-83a5-47d6-aa28-2f9fe56be8aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:03.553346 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b10a3-83a5-47d6-aa28-2f9fe56be8aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:03.555529 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b10a3-83a5-47d6-aa28-2f9fe56be8aa admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:03.566955 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:03.605799 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1385/2754] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:17:03 2022] POST /identity/v3/projects => generated 367 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:17:03.607613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a572a7-0836-4527-afd2-542d8a5c09d6 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:03.619873 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a572a7-0836-4527-afd2-542d8a5c09d6 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:03.631812 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:03.710540 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b104995-7e0f-4d68-879b-0a0db761f3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:03.711149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b104995-7e0f-4d68-879b-0a0db761f3ff admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:03.711470 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b104995-7e0f-4d68-879b-0a0db761f3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:03.711769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b104995-7e0f-4d68-879b-0a0db761f3ff admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:03.712397 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b104995-7e0f-4d68-879b-0a0db761f3ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:03.714355 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b104995-7e0f-4d68-879b-0a0db761f3ff admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:03.738948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1370/2755] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:17:03 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:17:03.745228 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b10a3-83a5-47d6-aa28-2f9fe56be8aa admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:03.760996 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b10a3-83a5-47d6-aa28-2f9fe56be8aa admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:03.767097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e65920-2b25-4988-96ad-ebb79681a9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:03.767728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e65920-2b25-4988-96ad-ebb79681a9af admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:03.767998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e65920-2b25-4988-96ad-ebb79681a9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:03.768293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e65920-2b25-4988-96ad-ebb79681a9af admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:03.768948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e65920-2b25-4988-96ad-ebb79681a9af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:03.770796 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e65920-2b25-4988-96ad-ebb79681a9af admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:03.782231 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1386/2756] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:17:03.787337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b104995-7e0f-4d68-879b-0a0db761f3ff admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:03.805762 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b104995-7e0f-4d68-879b-0a0db761f3ff admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:03.813459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b270c-9bb2-4651-9d68-7ff29be149d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:03.814871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b270c-9bb2-4651-9d68-7ff29be149d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:03.815328 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b270c-9bb2-4651-9d68-7ff29be149d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:03.815799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b270c-9bb2-4651-9d68-7ff29be149d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:03.816951 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b270c-9bb2-4651-9d68-7ff29be149d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:03.819888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b270c-9bb2-4651-9d68-7ff29be149d7 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:03.834727 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1371/2757] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:17:03.839083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e65920-2b25-4988-96ad-ebb79681a9af admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:03.850773 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e65920-2b25-4988-96ad-ebb79681a9af admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:03.856078 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df829fb1-44b8-4330-a6f0-70866b91a898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:03.856802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df829fb1-44b8-4330-a6f0-70866b91a898 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:03.857107 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df829fb1-44b8-4330-a6f0-70866b91a898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:03.857477 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df829fb1-44b8-4330-a6f0-70866b91a898 admin admin] PATH_INFO: `/v3/projects/e8d5d8c07a5441d1bc053d9334d23b6e/users/af68b6ae8aea45b6aa7577d227ac348b/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:03.879149 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df829fb1-44b8-4330-a6f0-70866b91a898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8d5d8c07a5441d1bc053d9334d23b6e, user_id=af68b6ae8aea45b6aa7577d227ac348b, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:03.882246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df829fb1-44b8-4330-a6f0-70866b91a898 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:03.905106 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1387/2758] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:17:03 2022] PUT /identity/v3/projects/e8d5d8c07a5441d1bc053d9334d23b6e/users/af68b6ae8aea45b6aa7577d227ac348b/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:03.915364 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97412-bc89-4250-aa55-998c8c9bf3c3 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:03.915643 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97412-bc89-4250-aa55-998c8c9bf3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:03.915832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97412-bc89-4250-aa55-998c8c9bf3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:03.946788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-4ac97412-bc89-4250-aa55-998c8c9bf3c3 None None] MFA Rules not processed for user `af68b6ae8aea45b6aa7577d227ac348b`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:03.948017 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-4ac97412-bc89-4250-aa55-998c8c9bf3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:04.034569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1372/2759] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:04.047527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df829fb1-44b8-4330-a6f0-70866b91a898 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:04.049762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-df829fb1-44b8-4330-a6f0-70866b91a898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:04.134095 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df829fb1-44b8-4330-a6f0-70866b91a898 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:04.138496 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d347f61-4d5a-47a8-bef9-2a355af776a7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d477a49a1724b7e8a4699a5ebd04bde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:04.139776 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d347f61-4d5a-47a8-bef9-2a355af776a7 service glance] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:04.140058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d347f61-4d5a-47a8-bef9-2a355af776a7 service glance] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:04.140513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d347f61-4d5a-47a8-bef9-2a355af776a7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:04.151705 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d347f61-4d5a-47a8-bef9-2a355af776a7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:04.153160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d347f61-4d5a-47a8-bef9-2a355af776a7 service glance] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:04.166099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1388/2760] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:17:05.601035 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac97412-bc89-4250-aa55-998c8c9bf3c3 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:05.602556 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-4ac97412-bc89-4250-aa55-998c8c9bf3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:05.672764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac97412-bc89-4250-aa55-998c8c9bf3c3 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:05.678290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307e5d5-e55b-4726-91d4-f385e4c3799a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:05.678975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307e5d5-e55b-4726-91d4-f385e4c3799a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:05.679253 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307e5d5-e55b-4726-91d4-f385e4c3799a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:05.679524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307e5d5-e55b-4726-91d4-f385e4c3799a admin admin] PATH_INFO: `/v3/users/9676597a1b884138bf5435f3b6cd63c1` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:05.680800 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307e5d5-e55b-4726-91d4-f385e4c3799a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9676597a1b884138bf5435f3b6cd63c1)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:05.682765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307e5d5-e55b-4726-91d4-f385e4c3799a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:05.707390 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:05.741974 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-6307e5d5-e55b-4726-91d4-f385e4c3799a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9676597a1b884138bf5435f3b6cd63c1'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:05.753024 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-6307e5d5-e55b-4726-91d4-f385e4c3799a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9676597a1b884138bf5435f3b6cd63c1'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:05.766457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-6307e5d5-e55b-4726-91d4-f385e4c3799a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9676597a1b884138bf5435f3b6cd63c1'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:05.775503 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-6307e5d5-e55b-4726-91d4-f385e4c3799a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9676597a1b884138bf5435f3b6cd63c1'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:05.776464 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-6307e5d5-e55b-4726-91d4-f385e4c3799a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9676597a1b884138bf5435f3b6cd63c1'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:05.787675 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1373/2761] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:17:05 2022] DELETE /identity/v3/users/9676597a1b884138bf5435f3b6cd63c1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:05.999053 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d347f61-4d5a-47a8-bef9-2a355af776a7 service glance] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:06.012118 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d347f61-4d5a-47a8-bef9-2a355af776a7 service glance] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:06.065062 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307e5d5-e55b-4726-91d4-f385e4c3799a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:06.066539 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6307e5d5-e55b-4726-91d4-f385e4c3799a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:06.086922 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73e3fa-ad5e-4e6e-8b73-769d795922d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:06.087973 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73e3fa-ad5e-4e6e-8b73-769d795922d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:06.088450 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73e3fa-ad5e-4e6e-8b73-769d795922d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:06.088835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73e3fa-ad5e-4e6e-8b73-769d795922d9 admin admin] PATH_INFO: `/v3/projects/002ffc44d77748ae9c618229e21f41e8` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:06.091464 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73e3fa-ad5e-4e6e-8b73-769d795922d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=002ffc44d77748ae9c618229e21f41e8)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:06.093971 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73e3fa-ad5e-4e6e-8b73-769d795922d9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:06.141840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307e5d5-e55b-4726-91d4-f385e4c3799a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:06.152633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4b73e3fa-ad5e-4e6e-8b73-769d795922d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '002ffc44d77748ae9c618229e21f41e8', 'request_id': 'req-4b73e3fa-ad5e-4e6e-8b73-769d795922d9'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:06.163310 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4b73e3fa-ad5e-4e6e-8b73-769d795922d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '002ffc44d77748ae9c618229e21f41e8', 'request_id': 'req-4b73e3fa-ad5e-4e6e-8b73-769d795922d9'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:06.165589 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:06.170207 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4b73e3fa-ad5e-4e6e-8b73-769d795922d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '002ffc44d77748ae9c618229e21f41e8', 'request_id': 'req-4b73e3fa-ad5e-4e6e-8b73-769d795922d9'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:06.184275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4b73e3fa-ad5e-4e6e-8b73-769d795922d9 admin admin] The token cache is being invalidate because project 002ffc44d77748ae9c618229e21f41e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:17:06.185125 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4b73e3fa-ad5e-4e6e-8b73-769d795922d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:06.185870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4b73e3fa-ad5e-4e6e-8b73-769d795922d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:06.187471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1389/2762] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:17:05 2022] DELETE /identity/v3/projects/002ffc44d77748ae9c618229e21f41e8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:06.193105 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b480024-d05f-4cff-b465-c4e631af322a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:06.193391 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b480024-d05f-4cff-b465-c4e631af322a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:06.193666 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b480024-d05f-4cff-b465-c4e631af322a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:06.210151 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff019aa-f1dd-45fb-8009-662407b2185b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:06.211422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff019aa-f1dd-45fb-8009-662407b2185b service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:06.211704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff019aa-f1dd-45fb-8009-662407b2185b service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:06.212144 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff019aa-f1dd-45fb-8009-662407b2185b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:06.213917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6ff019aa-f1dd-45fb-8009-662407b2185b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:06.216000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-6b480024-d05f-4cff-b465-c4e631af322a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:06.217182 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6b480024-d05f-4cff-b465-c4e631af322a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:06.230652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff019aa-f1dd-45fb-8009-662407b2185b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:06.232100 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff019aa-f1dd-45fb-8009-662407b2185b service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:06.240796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1374/2763] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:17:06.285550 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1390/2764] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:17:06.293765 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635f68a-c4fd-4eb4-b140-0cb040d2526f None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:06.294644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635f68a-c4fd-4eb4-b140-0cb040d2526f None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:06.295104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635f68a-c4fd-4eb4-b140-0cb040d2526f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:06.318216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-1635f68a-c4fd-4eb4-b140-0cb040d2526f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:06.319551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-1635f68a-c4fd-4eb4-b140-0cb040d2526f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:06.327933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1375/2765] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:06.333241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b480024-d05f-4cff-b465-c4e631af322a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:06.343907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b480024-d05f-4cff-b465-c4e631af322a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:06.350016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416f380-7822-4090-9f68-612d8a8bf7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:06.350723 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416f380-7822-4090-9f68-612d8a8bf7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:06.351087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416f380-7822-4090-9f68-612d8a8bf7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:06.351423 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416f380-7822-4090-9f68-612d8a8bf7b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:06.352083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416f380-7822-4090-9f68-612d8a8bf7b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:17:06.352384 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416f380-7822-4090-9f68-612d8a8bf7b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:06.353976 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416f380-7822-4090-9f68-612d8a8bf7b3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:06.365998 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1391/2766] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:17:06.370655 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635f68a-c4fd-4eb4-b140-0cb040d2526f None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:06.380161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635f68a-c4fd-4eb4-b140-0cb040d2526f None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:06.385060 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b1dce-dfc0-4072-9b89-b8176a9519bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:06.385663 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b1dce-dfc0-4072-9b89-b8176a9519bd admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:06.385939 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b1dce-dfc0-4072-9b89-b8176a9519bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:06.386260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b1dce-dfc0-4072-9b89-b8176a9519bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:06.386874 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b1dce-dfc0-4072-9b89-b8176a9519bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:06.388762 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b1dce-dfc0-4072-9b89-b8176a9519bd admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:06.427173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1376/2767] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:17:06 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:17:06.431459 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416f380-7822-4090-9f68-612d8a8bf7b3 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:06.441670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416f380-7822-4090-9f68-612d8a8bf7b3 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:06.505277 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708694e-2974-4608-992e-d8f3f85cf38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:06.505899 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708694e-2974-4608-992e-d8f3f85cf38c admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:06.506213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708694e-2974-4608-992e-d8f3f85cf38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:06.506490 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708694e-2974-4608-992e-d8f3f85cf38c admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:06.507127 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708694e-2974-4608-992e-d8f3f85cf38c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:06.509047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708694e-2974-4608-992e-d8f3f85cf38c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:06.538100 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1392/2768] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:17:06 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:17:06.544188 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b1dce-dfc0-4072-9b89-b8176a9519bd admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:06.557096 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b1dce-dfc0-4072-9b89-b8176a9519bd admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:06.561889 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b098218-142e-4248-8394-0fd131ad335a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:06.562638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b098218-142e-4248-8394-0fd131ad335a admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:06.562788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b098218-142e-4248-8394-0fd131ad335a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:06.563095 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b098218-142e-4248-8394-0fd131ad335a admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:06.564381 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b098218-142e-4248-8394-0fd131ad335a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:06.565994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b098218-142e-4248-8394-0fd131ad335a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:06.583763 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1377/2769] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:17:06.586926 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708694e-2974-4608-992e-d8f3f85cf38c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:06.597750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708694e-2974-4608-992e-d8f3f85cf38c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:06.602842 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8369272-333f-490c-ae2b-f1b38bcd70f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:06.603441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8369272-333f-490c-ae2b-f1b38bcd70f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:06.603834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8369272-333f-490c-ae2b-f1b38bcd70f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:06.604024 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8369272-333f-490c-ae2b-f1b38bcd70f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:06.604735 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8369272-333f-490c-ae2b-f1b38bcd70f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:06.606515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8369272-333f-490c-ae2b-f1b38bcd70f0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:06.619499 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1393/2770] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:17:06.623782 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b098218-142e-4248-8394-0fd131ad335a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:06.635702 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b098218-142e-4248-8394-0fd131ad335a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:06.642710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a1a11-249f-45a9-adb7-599dfb9f5a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:06.643498 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a1a11-249f-45a9-adb7-599dfb9f5a46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:06.643847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a1a11-249f-45a9-adb7-599dfb9f5a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:06.644187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a1a11-249f-45a9-adb7-599dfb9f5a46 admin admin] PATH_INFO: `/v3/projects/6ba16efe78f742a6bff2cb03b9d05c10/users/39b016dadb9f406c864ab5ca91f59ebf/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:06.659618 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a1a11-249f-45a9-adb7-599dfb9f5a46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ba16efe78f742a6bff2cb03b9d05c10, user_id=39b016dadb9f406c864ab5ca91f59ebf, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:06.662469 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a1a11-249f-45a9-adb7-599dfb9f5a46 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:06.665633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:06.686958 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1378/2771] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:17:06 2022] PUT /identity/v3/projects/6ba16efe78f742a6bff2cb03b9d05c10/users/39b016dadb9f406c864ab5ca91f59ebf/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:06.689681 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863f276-0875-45e9-af7b-6797c03bfef3 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:06.689982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863f276-0875-45e9-af7b-6797c03bfef3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:06.690397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863f276-0875-45e9-af7b-6797c03bfef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:06.731343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-0863f276-0875-45e9-af7b-6797c03bfef3 None None] MFA Rules not processed for user `39b016dadb9f406c864ab5ca91f59ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:06.733157 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0863f276-0875-45e9-af7b-6797c03bfef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:06.792794 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:06.828371 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1394/2772] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:06.835104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a1a11-249f-45a9-adb7-599dfb9f5a46 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:06.914847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a1a11-249f-45a9-adb7-599dfb9f5a46 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:06.920619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766819e-05e2-4b8a-beb3-629e5d71e389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:06.921239 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766819e-05e2-4b8a-beb3-629e5d71e389 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:06.921515 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766819e-05e2-4b8a-beb3-629e5d71e389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:06.921779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766819e-05e2-4b8a-beb3-629e5d71e389 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:06.922443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766819e-05e2-4b8a-beb3-629e5d71e389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:06.924327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766819e-05e2-4b8a-beb3-629e5d71e389 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:06.954380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1379/2773] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:17:06 2022] POST /identity/v3/projects => generated 367 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:17:06.957920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863f276-0875-45e9-af7b-6797c03bfef3 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:06.986185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863f276-0875-45e9-af7b-6797c03bfef3 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:07.059236 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa2dbb-229f-49ff-a7fc-73bc64f80944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:07.059934 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa2dbb-229f-49ff-a7fc-73bc64f80944 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:07.060124 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa2dbb-229f-49ff-a7fc-73bc64f80944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:07.060398 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa2dbb-229f-49ff-a7fc-73bc64f80944 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:07.061039 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa2dbb-229f-49ff-a7fc-73bc64f80944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:07.062988 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa2dbb-229f-49ff-a7fc-73bc64f80944 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:07.087828 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1395/2774] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:17:06 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:17:07.092069 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766819e-05e2-4b8a-beb3-629e5d71e389 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:07.103026 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766819e-05e2-4b8a-beb3-629e5d71e389 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:07.107837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07857297-d0a7-44ed-b492-c3077e6f2372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:07.108503 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07857297-d0a7-44ed-b492-c3077e6f2372 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:07.108786 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07857297-d0a7-44ed-b492-c3077e6f2372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:07.109052 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07857297-d0a7-44ed-b492-c3077e6f2372 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:07.109735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07857297-d0a7-44ed-b492-c3077e6f2372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:07.112312 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07857297-d0a7-44ed-b492-c3077e6f2372 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:07.124829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1380/2775] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:17:07.130343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa2dbb-229f-49ff-a7fc-73bc64f80944 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:07.141442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa2dbb-229f-49ff-a7fc-73bc64f80944 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:07.147035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c430568-dfa3-41da-88fe-d76e372547b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:07.147705 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c430568-dfa3-41da-88fe-d76e372547b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:07.147982 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c430568-dfa3-41da-88fe-d76e372547b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:07.148253 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c430568-dfa3-41da-88fe-d76e372547b6 admin admin] PATH_INFO: `/v3/projects/1c27366280fd4ef7bf04303a25725cac/users/3106fbf516ce4753b72f2ea5954e6175/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:07.165156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c430568-dfa3-41da-88fe-d76e372547b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c27366280fd4ef7bf04303a25725cac, user_id=3106fbf516ce4753b72f2ea5954e6175, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:07.167852 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c430568-dfa3-41da-88fe-d76e372547b6 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:07.183964 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1396/2776] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:17:07 2022] PUT /identity/v3/projects/1c27366280fd4ef7bf04303a25725cac/users/3106fbf516ce4753b72f2ea5954e6175/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:07.187793 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07857297-d0a7-44ed-b492-c3077e6f2372 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:07.208484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07857297-d0a7-44ed-b492-c3077e6f2372 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:07.274657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b69c97-37b5-4797-9878-bafd29d7ef4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:07.275269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b69c97-37b5-4797-9878-bafd29d7ef4b admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:07.275544 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b69c97-37b5-4797-9878-bafd29d7ef4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:07.275813 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b69c97-37b5-4797-9878-bafd29d7ef4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:07.276459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b69c97-37b5-4797-9878-bafd29d7ef4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:07.278107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b69c97-37b5-4797-9878-bafd29d7ef4b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:07.296354 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1381/2777] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:17:07.299558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c430568-dfa3-41da-88fe-d76e372547b6 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:07.310397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c430568-dfa3-41da-88fe-d76e372547b6 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:07.316452 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfea88-662f-49d3-9eb9-920f9ef671a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:07.317275 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfea88-662f-49d3-9eb9-920f9ef671a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:07.317661 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfea88-662f-49d3-9eb9-920f9ef671a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:07.318234 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfea88-662f-49d3-9eb9-920f9ef671a7 admin admin] PATH_INFO: `/v3/projects/1c27366280fd4ef7bf04303a25725cac/users/3106fbf516ce4753b72f2ea5954e6175/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:07.320529 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfea88-662f-49d3-9eb9-920f9ef671a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c27366280fd4ef7bf04303a25725cac, user_id=3106fbf516ce4753b72f2ea5954e6175, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:07.325158 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfea88-662f-49d3-9eb9-920f9ef671a7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:07.335608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1397/2778] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:17:07 2022] PUT /identity/v3/projects/1c27366280fd4ef7bf04303a25725cac/users/3106fbf516ce4753b72f2ea5954e6175/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:17:07.347893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1048868-b4e6-40a9-96b3-25e19c60afec None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:07.348262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1048868-b4e6-40a9-96b3-25e19c60afec None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:07.348725 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1048868-b4e6-40a9-96b3-25e19c60afec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:07.386437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f1048868-b4e6-40a9-96b3-25e19c60afec None None] MFA Rules not processed for user `3106fbf516ce4753b72f2ea5954e6175`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:07.387848 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f1048868-b4e6-40a9-96b3-25e19c60afec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:07.446461 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:07.468015 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1382/2779] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:07.479844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfea88-662f-49d3-9eb9-920f9ef671a7 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:07.482389 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-39cfea88-662f-49d3-9eb9-920f9ef671a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:07.548743 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfea88-662f-49d3-9eb9-920f9ef671a7 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:07.553755 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be4285-8d4b-4e50-a4ba-368b22e30fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:07.554531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be4285-8d4b-4e50-a4ba-368b22e30fbd service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:07.554859 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be4285-8d4b-4e50-a4ba-368b22e30fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:07.555135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be4285-8d4b-4e50-a4ba-368b22e30fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:07.566135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be4285-8d4b-4e50-a4ba-368b22e30fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:07.567574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be4285-8d4b-4e50-a4ba-368b22e30fbd service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:07.578519 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1398/2780] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:17:07.627623 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1048868-b4e6-40a9-96b3-25e19c60afec None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:07.629055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f1048868-b4e6-40a9-96b3-25e19c60afec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:07.641351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1048868-b4e6-40a9-96b3-25e19c60afec None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:07.645594 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc543b7-21b2-4754-afed-c9d71bbd9819 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:07.646230 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc543b7-21b2-4754-afed-c9d71bbd9819 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:07.646513 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc543b7-21b2-4754-afed-c9d71bbd9819 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:07.646781 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc543b7-21b2-4754-afed-c9d71bbd9819 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:07.707648 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc543b7-21b2-4754-afed-c9d71bbd9819 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:07.709080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc543b7-21b2-4754-afed-c9d71bbd9819 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:07.718235 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1383/2781] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:17:11.088378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be4285-8d4b-4e50-a4ba-368b22e30fbd service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:11.137518 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:11.170603 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be4285-8d4b-4e50-a4ba-368b22e30fbd service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:11.175255 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e1c18-65c3-4842-9f8a-df901a9b72ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:11.175993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e1c18-65c3-4842-9f8a-df901a9b72ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:11.176351 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e1c18-65c3-4842-9f8a-df901a9b72ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:11.176683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e1c18-65c3-4842-9f8a-df901a9b72ec admin admin] PATH_INFO: `/v3/users/39b016dadb9f406c864ab5ca91f59ebf` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:11.177908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e1c18-65c3-4842-9f8a-df901a9b72ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=39b016dadb9f406c864ab5ca91f59ebf)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:11.179902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e1c18-65c3-4842-9f8a-df901a9b72ec admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:11.232558 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-749e1c18-65c3-4842-9f8a-df901a9b72ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39b016dadb9f406c864ab5ca91f59ebf'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:11.233268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-749e1c18-65c3-4842-9f8a-df901a9b72ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39b016dadb9f406c864ab5ca91f59ebf'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:11.240520 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-749e1c18-65c3-4842-9f8a-df901a9b72ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39b016dadb9f406c864ab5ca91f59ebf'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:11.248119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-749e1c18-65c3-4842-9f8a-df901a9b72ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39b016dadb9f406c864ab5ca91f59ebf'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:11.261131 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-749e1c18-65c3-4842-9f8a-df901a9b72ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39b016dadb9f406c864ab5ca91f59ebf'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:11.267711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1399/2782] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:17:11 2022] DELETE /identity/v3/users/39b016dadb9f406c864ab5ca91f59ebf => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:11.272875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8455c-15c7-4ca6-a73f-7cfb24ddc9ac None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:11.273383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8455c-15c7-4ca6-a73f-7cfb24ddc9ac None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:11.273570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8455c-15c7-4ca6-a73f-7cfb24ddc9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:11.294102 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-cae8455c-15c7-4ca6-a73f-7cfb24ddc9ac None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:11.295352 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-cae8455c-15c7-4ca6-a73f-7cfb24ddc9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:11.357778 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1384/2783] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:11.368158 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e1c18-65c3-4842-9f8a-df901a9b72ec admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:11.369835 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-749e1c18-65c3-4842-9f8a-df901a9b72ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:11.436525 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e1c18-65c3-4842-9f8a-df901a9b72ec admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:11.440737 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4febfb-b46a-4701-86e2-ba6b16c050ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:11.441345 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4febfb-b46a-4701-86e2-ba6b16c050ef service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:11.441620 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4febfb-b46a-4701-86e2-ba6b16c050ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:11.441919 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4febfb-b46a-4701-86e2-ba6b16c050ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:11.450841 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4febfb-b46a-4701-86e2-ba6b16c050ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:11.452244 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4febfb-b46a-4701-86e2-ba6b16c050ef service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:11.462312 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1400/2784] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:17:11.740333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8455c-15c7-4ca6-a73f-7cfb24ddc9ac None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:11.741895 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-cae8455c-15c7-4ca6-a73f-7cfb24ddc9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:11.755704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8455c-15c7-4ca6-a73f-7cfb24ddc9ac None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:11.765122 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7cb2e-dc56-42ac-af44-9d670d6839f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:11.765848 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7cb2e-dc56-42ac-af44-9d670d6839f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:11.766257 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7cb2e-dc56-42ac-af44-9d670d6839f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:11.766504 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7cb2e-dc56-42ac-af44-9d670d6839f0 admin admin] PATH_INFO: `/v3/projects/6ba16efe78f742a6bff2cb03b9d05c10` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:11.768839 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7cb2e-dc56-42ac-af44-9d670d6839f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ba16efe78f742a6bff2cb03b9d05c10)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:11.770951 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7cb2e-dc56-42ac-af44-9d670d6839f0 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:11.826653 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-67e7cb2e-dc56-42ac-af44-9d670d6839f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ba16efe78f742a6bff2cb03b9d05c10', 'request_id': 'req-67e7cb2e-dc56-42ac-af44-9d670d6839f0'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:11.834188 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-67e7cb2e-dc56-42ac-af44-9d670d6839f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ba16efe78f742a6bff2cb03b9d05c10', 'request_id': 'req-67e7cb2e-dc56-42ac-af44-9d670d6839f0'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:11.841205 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-67e7cb2e-dc56-42ac-af44-9d670d6839f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ba16efe78f742a6bff2cb03b9d05c10', 'request_id': 'req-67e7cb2e-dc56-42ac-af44-9d670d6839f0'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:11.853488 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-67e7cb2e-dc56-42ac-af44-9d670d6839f0 admin admin] The token cache is being invalidate because project 6ba16efe78f742a6bff2cb03b9d05c10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:17:11.853818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-67e7cb2e-dc56-42ac-af44-9d670d6839f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:11.854527 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-67e7cb2e-dc56-42ac-af44-9d670d6839f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:11.856013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1385/2785] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:17:11 2022] DELETE /identity/v3/projects/6ba16efe78f742a6bff2cb03b9d05c10 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:11.860036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4febfb-b46a-4701-86e2-ba6b16c050ef service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:11.870783 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4febfb-b46a-4701-86e2-ba6b16c050ef service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:11.930737 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b81ec8-6166-4bb7-ae35-f38428dfc724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:11.931365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b81ec8-6166-4bb7-ae35-f38428dfc724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:11.931655 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b81ec8-6166-4bb7-ae35-f38428dfc724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:11.931910 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b81ec8-6166-4bb7-ae35-f38428dfc724 admin admin] PATH_INFO: `/v3/users/3106fbf516ce4753b72f2ea5954e6175` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:11.933027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b81ec8-6166-4bb7-ae35-f38428dfc724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3106fbf516ce4753b72f2ea5954e6175)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:11.934978 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b81ec8-6166-4bb7-ae35-f38428dfc724 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:11.936416 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:11.987902 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-05b81ec8-6166-4bb7-ae35-f38428dfc724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3106fbf516ce4753b72f2ea5954e6175'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:11.988563 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-05b81ec8-6166-4bb7-ae35-f38428dfc724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3106fbf516ce4753b72f2ea5954e6175'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:11.996642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-05b81ec8-6166-4bb7-ae35-f38428dfc724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3106fbf516ce4753b72f2ea5954e6175'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:12.004871 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-05b81ec8-6166-4bb7-ae35-f38428dfc724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3106fbf516ce4753b72f2ea5954e6175'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:12.017986 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-05b81ec8-6166-4bb7-ae35-f38428dfc724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3106fbf516ce4753b72f2ea5954e6175'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:12.025911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1401/2786] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:17:11 2022] DELETE /identity/v3/users/3106fbf516ce4753b72f2ea5954e6175 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:12.318560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7cb2e-dc56-42ac-af44-9d670d6839f0 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:12.320341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-67e7cb2e-dc56-42ac-af44-9d670d6839f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:12.336546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7cb2e-dc56-42ac-af44-9d670d6839f0 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:12.347391 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:12.399458 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4c5c4-e35d-4055-a7f1-c6da68c3c6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:12.400229 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4c5c4-e35d-4055-a7f1-c6da68c3c6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:12.400500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4c5c4-e35d-4055-a7f1-c6da68c3c6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:12.400766 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4c5c4-e35d-4055-a7f1-c6da68c3c6b8 admin admin] PATH_INFO: `/v3/projects/1c27366280fd4ef7bf04303a25725cac` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:12.401940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4c5c4-e35d-4055-a7f1-c6da68c3c6b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c27366280fd4ef7bf04303a25725cac)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:12.404520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4c5c4-e35d-4055-a7f1-c6da68c3c6b8 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:12.452331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-e6c4c5c4-e35d-4055-a7f1-c6da68c3c6b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c27366280fd4ef7bf04303a25725cac', 'request_id': 'req-e6c4c5c4-e35d-4055-a7f1-c6da68c3c6b8'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:12.466332 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-e6c4c5c4-e35d-4055-a7f1-c6da68c3c6b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c27366280fd4ef7bf04303a25725cac', 'request_id': 'req-e6c4c5c4-e35d-4055-a7f1-c6da68c3c6b8'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:12.473226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-e6c4c5c4-e35d-4055-a7f1-c6da68c3c6b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c27366280fd4ef7bf04303a25725cac', 'request_id': 'req-e6c4c5c4-e35d-4055-a7f1-c6da68c3c6b8'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:12.485562 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-e6c4c5c4-e35d-4055-a7f1-c6da68c3c6b8 admin admin] The token cache is being invalidate because project 1c27366280fd4ef7bf04303a25725cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:17:12.485852 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-e6c4c5c4-e35d-4055-a7f1-c6da68c3c6b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:12.486532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-e6c4c5c4-e35d-4055-a7f1-c6da68c3c6b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:12.488040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1386/2787] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:17:12 2022] DELETE /identity/v3/projects/1c27366280fd4ef7bf04303a25725cac => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:12.493144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeeed59-8c0c-4cf3-98aa-2c9835ad149f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:12.493646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeeed59-8c0c-4cf3-98aa-2c9835ad149f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:12.493766 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeeed59-8c0c-4cf3-98aa-2c9835ad149f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:12.514397 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-bfeeed59-8c0c-4cf3-98aa-2c9835ad149f None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:12.515767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bfeeed59-8c0c-4cf3-98aa-2c9835ad149f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:12.590331 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1402/2788] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:12.592654 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194202da-3ede-4bbb-bea1-82b1721b6378 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:12.592959 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194202da-3ede-4bbb-bea1-82b1721b6378 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:12.593267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194202da-3ede-4bbb-bea1-82b1721b6378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:12.618545 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-194202da-3ede-4bbb-bea1-82b1721b6378 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:12.620012 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-194202da-3ede-4bbb-bea1-82b1721b6378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:12.629030 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1387/2789] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:12.633467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeeed59-8c0c-4cf3-98aa-2c9835ad149f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:12.645582 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeeed59-8c0c-4cf3-98aa-2c9835ad149f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:12.650430 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085aeacf-d20e-4a1e-b902-b6b5763e7dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:12.651050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085aeacf-d20e-4a1e-b902-b6b5763e7dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:12.651328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085aeacf-d20e-4a1e-b902-b6b5763e7dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:12.651600 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085aeacf-d20e-4a1e-b902-b6b5763e7dd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:12.652248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085aeacf-d20e-4a1e-b902-b6b5763e7dd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:17:12.652565 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085aeacf-d20e-4a1e-b902-b6b5763e7dd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:12.654166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085aeacf-d20e-4a1e-b902-b6b5763e7dd1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:12.666981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1403/2790] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:17:12.671105 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194202da-3ede-4bbb-bea1-82b1721b6378 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:12.680657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194202da-3ede-4bbb-bea1-82b1721b6378 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:12.687915 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbe9f1-41c0-4a4a-bd47-80fed5439890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:12.689158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbe9f1-41c0-4a4a-bd47-80fed5439890 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:12.689452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbe9f1-41c0-4a4a-bd47-80fed5439890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:12.689734 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbe9f1-41c0-4a4a-bd47-80fed5439890 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:12.690423 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bbe9f1-41c0-4a4a-bd47-80fed5439890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:12.692344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bbe9f1-41c0-4a4a-bd47-80fed5439890 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:12.723174 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1388/2791] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:17:12 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:17:12.727597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085aeacf-d20e-4a1e-b902-b6b5763e7dd1 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:12.741008 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085aeacf-d20e-4a1e-b902-b6b5763e7dd1 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:12.809258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c93af9-1f3c-4b3b-bad3-c0d47e929a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:12.810044 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c93af9-1f3c-4b3b-bad3-c0d47e929a77 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:12.810460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c93af9-1f3c-4b3b-bad3-c0d47e929a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:12.810843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c93af9-1f3c-4b3b-bad3-c0d47e929a77 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:12.811666 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c93af9-1f3c-4b3b-bad3-c0d47e929a77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:12.815098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c93af9-1f3c-4b3b-bad3-c0d47e929a77 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:12.827393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:12.840719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1404/2792] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:17:12 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:17:12.844639 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbe9f1-41c0-4a4a-bd47-80fed5439890 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:12.856077 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbe9f1-41c0-4a4a-bd47-80fed5439890 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:12.861149 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433882e-b84a-4ad2-abb7-11155b19c0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:12.861703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433882e-b84a-4ad2-abb7-11155b19c0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:12.862031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433882e-b84a-4ad2-abb7-11155b19c0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:12.862344 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433882e-b84a-4ad2-abb7-11155b19c0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:12.863046 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433882e-b84a-4ad2-abb7-11155b19c0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:12.864701 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433882e-b84a-4ad2-abb7-11155b19c0b7 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:12.880152 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1389/2793] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:17:12.884360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c93af9-1f3c-4b3b-bad3-c0d47e929a77 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:12.895667 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c93af9-1f3c-4b3b-bad3-c0d47e929a77 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:12.900825 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0228b-908c-40b0-9895-b891853df397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:12.901437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0228b-908c-40b0-9895-b891853df397 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:12.901718 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0228b-908c-40b0-9895-b891853df397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:12.901995 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0228b-908c-40b0-9895-b891853df397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:12.902670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d0228b-908c-40b0-9895-b891853df397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:12.904268 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d0228b-908c-40b0-9895-b891853df397 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:12.917069 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1405/2794] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:17:12.921729 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433882e-b84a-4ad2-abb7-11155b19c0b7 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:12.933495 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433882e-b84a-4ad2-abb7-11155b19c0b7 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:12.938061 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef44e6f-f8b7-4d4f-9e18-3f4e4570534b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:12.939248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef44e6f-f8b7-4d4f-9e18-3f4e4570534b admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:12.939523 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef44e6f-f8b7-4d4f-9e18-3f4e4570534b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:12.939790 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef44e6f-f8b7-4d4f-9e18-3f4e4570534b admin admin] PATH_INFO: `/v3/projects/db82e3e9b0d24ab7ad9c99c527e70525/users/95b50c59bc1f41ab82477de0025e5c25/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:12.955294 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef44e6f-f8b7-4d4f-9e18-3f4e4570534b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db82e3e9b0d24ab7ad9c99c527e70525, user_id=95b50c59bc1f41ab82477de0025e5c25, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:12.957942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef44e6f-f8b7-4d4f-9e18-3f4e4570534b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:12.970824 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1390/2795] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:17:12 2022] PUT /identity/v3/projects/db82e3e9b0d24ab7ad9c99c527e70525/users/95b50c59bc1f41ab82477de0025e5c25/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:12.983273 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95344593-319f-43e2-9e56-b124f9dac718 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:12.983599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95344593-319f-43e2-9e56-b124f9dac718 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:12.983927 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95344593-319f-43e2-9e56-b124f9dac718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:13.012817 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-95344593-319f-43e2-9e56-b124f9dac718 None None] MFA Rules not processed for user `95b50c59bc1f41ab82477de0025e5c25`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:13.014030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-95344593-319f-43e2-9e56-b124f9dac718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:13.088192 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1406/2796] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:13.092784 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef44e6f-f8b7-4d4f-9e18-3f4e4570534b admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:13.136037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:13.156882 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef44e6f-f8b7-4d4f-9e18-3f4e4570534b admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:13.161672 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e66977-ea21-4c1c-9c7e-b26ddb516d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:13.162301 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e66977-ea21-4c1c-9c7e-b26ddb516d42 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:13.162584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e66977-ea21-4c1c-9c7e-b26ddb516d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:13.162850 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e66977-ea21-4c1c-9c7e-b26ddb516d42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:13.163479 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e66977-ea21-4c1c-9c7e-b26ddb516d42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:13.165451 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e66977-ea21-4c1c-9c7e-b26ddb516d42 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:13.196147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1391/2797] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:17:13 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:17:13.200055 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95344593-319f-43e2-9e56-b124f9dac718 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:13.212018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95344593-319f-43e2-9e56-b124f9dac718 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:13.279996 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4b889-5aae-4eab-9350-8bf205ae712d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:13.280649 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4b889-5aae-4eab-9350-8bf205ae712d admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:13.280909 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4b889-5aae-4eab-9350-8bf205ae712d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:13.281178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4b889-5aae-4eab-9350-8bf205ae712d admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:13.281829 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4b889-5aae-4eab-9350-8bf205ae712d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:13.283789 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4b889-5aae-4eab-9350-8bf205ae712d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:13.310155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1407/2798] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:17:13 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:17:13.313158 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e66977-ea21-4c1c-9c7e-b26ddb516d42 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:13.326737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e66977-ea21-4c1c-9c7e-b26ddb516d42 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:13.332454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756a3d8-1e13-44f0-a5a4-872bce9807e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:13.333058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756a3d8-1e13-44f0-a5a4-872bce9807e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:13.333337 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756a3d8-1e13-44f0-a5a4-872bce9807e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:13.333608 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756a3d8-1e13-44f0-a5a4-872bce9807e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:13.334285 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756a3d8-1e13-44f0-a5a4-872bce9807e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:13.335869 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756a3d8-1e13-44f0-a5a4-872bce9807e2 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:13.347798 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1392/2799] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:17:13.351938 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4b889-5aae-4eab-9350-8bf205ae712d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:13.361881 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4b889-5aae-4eab-9350-8bf205ae712d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:13.366737 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e87071-1441-4b72-815a-519ba08f98fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:13.367409 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e87071-1441-4b72-815a-519ba08f98fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:13.367708 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e87071-1441-4b72-815a-519ba08f98fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:13.367980 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e87071-1441-4b72-815a-519ba08f98fd admin admin] PATH_INFO: `/v3/projects/d072351d1e014772891772d4a6989d92/users/24393c7e8e194c8097270e173c59029b/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:13.383921 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e87071-1441-4b72-815a-519ba08f98fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d072351d1e014772891772d4a6989d92, user_id=24393c7e8e194c8097270e173c59029b, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:13.386678 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e87071-1441-4b72-815a-519ba08f98fd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:13.401612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1408/2800] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:17:13 2022] PUT /identity/v3/projects/d072351d1e014772891772d4a6989d92/users/24393c7e8e194c8097270e173c59029b/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:13.405692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756a3d8-1e13-44f0-a5a4-872bce9807e2 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:13.415888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756a3d8-1e13-44f0-a5a4-872bce9807e2 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:13.474350 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68500b-70a9-40f6-b427-67ad924ccea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:13.474996 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68500b-70a9-40f6-b427-67ad924ccea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:13.475359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68500b-70a9-40f6-b427-67ad924ccea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:13.475651 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68500b-70a9-40f6-b427-67ad924ccea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:13.476484 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68500b-70a9-40f6-b427-67ad924ccea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:13.477984 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68500b-70a9-40f6-b427-67ad924ccea0 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:13.488385 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1393/2801] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:17:13.492551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e87071-1441-4b72-815a-519ba08f98fd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:13.504863 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e87071-1441-4b72-815a-519ba08f98fd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:13.509471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c2e2d-d02d-46af-af0a-450a5d3fa969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:13.510136 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c2e2d-d02d-46af-af0a-450a5d3fa969 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:13.510427 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c2e2d-d02d-46af-af0a-450a5d3fa969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:13.510696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c2e2d-d02d-46af-af0a-450a5d3fa969 admin admin] PATH_INFO: `/v3/projects/d072351d1e014772891772d4a6989d92/users/24393c7e8e194c8097270e173c59029b/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:13.512583 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c2e2d-d02d-46af-af0a-450a5d3fa969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d072351d1e014772891772d4a6989d92, user_id=24393c7e8e194c8097270e173c59029b, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:13.515213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c2e2d-d02d-46af-af0a-450a5d3fa969 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:13.526013 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1409/2802] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:17:13 2022] PUT /identity/v3/projects/d072351d1e014772891772d4a6989d92/users/24393c7e8e194c8097270e173c59029b/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:17:13.537450 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ab3c2-2ae4-454b-bce7-e2c7d0ad88eb None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:13.537750 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ab3c2-2ae4-454b-bce7-e2c7d0ad88eb None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:13.538049 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ab3c2-2ae4-454b-bce7-e2c7d0ad88eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:13.566449 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-275ab3c2-2ae4-454b-bce7-e2c7d0ad88eb None None] MFA Rules not processed for user `24393c7e8e194c8097270e173c59029b`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:13.567677 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-275ab3c2-2ae4-454b-bce7-e2c7d0ad88eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:13.646298 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1394/2803] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:13 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:13.657393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c2e2d-d02d-46af-af0a-450a5d3fa969 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:13.659400 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1a3c2e2d-d02d-46af-af0a-450a5d3fa969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:13.691311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:13.721944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c2e2d-d02d-46af-af0a-450a5d3fa969 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:13.725693 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711445f-75ec-41d1-b0cc-767bdfb28755 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:13.726338 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711445f-75ec-41d1-b0cc-767bdfb28755 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:13.726619 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711445f-75ec-41d1-b0cc-767bdfb28755 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:13.726925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711445f-75ec-41d1-b0cc-767bdfb28755 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:13.737573 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711445f-75ec-41d1-b0cc-767bdfb28755 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:13.739034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711445f-75ec-41d1-b0cc-767bdfb28755 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:13.748809 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1410/2804] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:17:14.128569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ab3c2-2ae4-454b-bce7-e2c7d0ad88eb None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:14.168146 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:14.193436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ab3c2-2ae4-454b-bce7-e2c7d0ad88eb None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:14.198570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded7940-c488-425d-ad60-ec04c46277cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:14.199333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded7940-c488-425d-ad60-ec04c46277cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:14.199619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded7940-c488-425d-ad60-ec04c46277cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:14.199893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded7940-c488-425d-ad60-ec04c46277cd admin admin] PATH_INFO: `/v3/users/95b50c59bc1f41ab82477de0025e5c25` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:14.201051 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded7940-c488-425d-ad60-ec04c46277cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=95b50c59bc1f41ab82477de0025e5c25)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:14.203081 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded7940-c488-425d-ad60-ec04c46277cd admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:14.252465 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-6ded7940-c488-425d-ad60-ec04c46277cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95b50c59bc1f41ab82477de0025e5c25'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:14.259504 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-6ded7940-c488-425d-ad60-ec04c46277cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95b50c59bc1f41ab82477de0025e5c25'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:14.271962 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-6ded7940-c488-425d-ad60-ec04c46277cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95b50c59bc1f41ab82477de0025e5c25'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:14.278847 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-6ded7940-c488-425d-ad60-ec04c46277cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95b50c59bc1f41ab82477de0025e5c25'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:14.279508 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-6ded7940-c488-425d-ad60-ec04c46277cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95b50c59bc1f41ab82477de0025e5c25'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:14.285567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1395/2805] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:17:14 2022] DELETE /identity/v3/users/95b50c59bc1f41ab82477de0025e5c25 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:14.290083 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518f742-7140-423e-a850-b9acabd27352 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:14.290392 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518f742-7140-423e-a850-b9acabd27352 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:14.290663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518f742-7140-423e-a850-b9acabd27352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:14.312629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2518f742-7140-423e-a850-b9acabd27352 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:14.315598 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2518f742-7140-423e-a850-b9acabd27352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:14.382413 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1411/2806] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:14.392802 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded7940-c488-425d-ad60-ec04c46277cd admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:14.394336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6ded7940-c488-425d-ad60-ec04c46277cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:14.453783 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded7940-c488-425d-ad60-ec04c46277cd admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:14.458137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd846f0-56b6-47c9-b857-606f352037ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:14.458767 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd846f0-56b6-47c9-b857-606f352037ea service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:14.459048 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd846f0-56b6-47c9-b857-606f352037ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:14.459317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd846f0-56b6-47c9-b857-606f352037ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:14.469274 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd846f0-56b6-47c9-b857-606f352037ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:14.470843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd846f0-56b6-47c9-b857-606f352037ea service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:14.483922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1396/2807] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:17:15.023310 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518f742-7140-423e-a850-b9acabd27352 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:15.024845 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2518f742-7140-423e-a850-b9acabd27352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:15.036983 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518f742-7140-423e-a850-b9acabd27352 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:15.042527 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9574b0-0f75-4411-87c7-f58acc75c905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:15.043253 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9574b0-0f75-4411-87c7-f58acc75c905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:15.043601 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9574b0-0f75-4411-87c7-f58acc75c905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:15.043944 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9574b0-0f75-4411-87c7-f58acc75c905 admin admin] PATH_INFO: `/v3/projects/db82e3e9b0d24ab7ad9c99c527e70525` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:15.045156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9574b0-0f75-4411-87c7-f58acc75c905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db82e3e9b0d24ab7ad9c99c527e70525)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:15.047196 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9574b0-0f75-4411-87c7-f58acc75c905 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:15.096143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2d9574b0-0f75-4411-87c7-f58acc75c905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db82e3e9b0d24ab7ad9c99c527e70525', 'request_id': 'req-2d9574b0-0f75-4411-87c7-f58acc75c905'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:15.103911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2d9574b0-0f75-4411-87c7-f58acc75c905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db82e3e9b0d24ab7ad9c99c527e70525', 'request_id': 'req-2d9574b0-0f75-4411-87c7-f58acc75c905'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:15.109451 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2d9574b0-0f75-4411-87c7-f58acc75c905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db82e3e9b0d24ab7ad9c99c527e70525', 'request_id': 'req-2d9574b0-0f75-4411-87c7-f58acc75c905'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:15.120726 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2d9574b0-0f75-4411-87c7-f58acc75c905 admin admin] The token cache is being invalidate because project db82e3e9b0d24ab7ad9c99c527e70525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:17:15.121008 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2d9574b0-0f75-4411-87c7-f58acc75c905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:15.121677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2d9574b0-0f75-4411-87c7-f58acc75c905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:15.123204 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1412/2808] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:17:15 2022] DELETE /identity/v3/projects/db82e3e9b0d24ab7ad9c99c527e70525 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:17:15.128098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd846f0-56b6-47c9-b857-606f352037ea service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:15.137732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd846f0-56b6-47c9-b857-606f352037ea service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:15.198239 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:15.207487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e910a-beca-4044-8009-3cd06fb76e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:15.208182 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e910a-beca-4044-8009-3cd06fb76e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:15.208674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e910a-beca-4044-8009-3cd06fb76e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:15.209014 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e910a-beca-4044-8009-3cd06fb76e88 admin admin] PATH_INFO: `/v3/users/24393c7e8e194c8097270e173c59029b` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:15.210411 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e910a-beca-4044-8009-3cd06fb76e88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24393c7e8e194c8097270e173c59029b)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:15.212917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e910a-beca-4044-8009-3cd06fb76e88 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:15.264889 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ab1e910a-beca-4044-8009-3cd06fb76e88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24393c7e8e194c8097270e173c59029b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:15.271843 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ab1e910a-beca-4044-8009-3cd06fb76e88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24393c7e8e194c8097270e173c59029b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:15.283875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ab1e910a-beca-4044-8009-3cd06fb76e88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24393c7e8e194c8097270e173c59029b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:15.290776 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ab1e910a-beca-4044-8009-3cd06fb76e88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24393c7e8e194c8097270e173c59029b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:15.291413 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-ab1e910a-beca-4044-8009-3cd06fb76e88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24393c7e8e194c8097270e173c59029b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:15.297217 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1397/2809] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:17:15 2022] DELETE /identity/v3/users/24393c7e8e194c8097270e173c59029b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:15.571896 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9574b0-0f75-4411-87c7-f58acc75c905 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:15.573448 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2d9574b0-0f75-4411-87c7-f58acc75c905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:15.586919 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9574b0-0f75-4411-87c7-f58acc75c905 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:15.637341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:15.648352 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fef69-832b-4f27-8b63-d48bb2561ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:15.649015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fef69-832b-4f27-8b63-d48bb2561ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:15.649292 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fef69-832b-4f27-8b63-d48bb2561ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:15.649565 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fef69-832b-4f27-8b63-d48bb2561ccd admin admin] PATH_INFO: `/v3/projects/d072351d1e014772891772d4a6989d92` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:15.650754 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fef69-832b-4f27-8b63-d48bb2561ccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d072351d1e014772891772d4a6989d92)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:15.652700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fef69-832b-4f27-8b63-d48bb2561ccd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:15.698635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-278fef69-832b-4f27-8b63-d48bb2561ccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd072351d1e014772891772d4a6989d92', 'request_id': 'req-278fef69-832b-4f27-8b63-d48bb2561ccd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:15.706431 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-278fef69-832b-4f27-8b63-d48bb2561ccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd072351d1e014772891772d4a6989d92', 'request_id': 'req-278fef69-832b-4f27-8b63-d48bb2561ccd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:15.712096 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-278fef69-832b-4f27-8b63-d48bb2561ccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd072351d1e014772891772d4a6989d92', 'request_id': 'req-278fef69-832b-4f27-8b63-d48bb2561ccd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:15.723662 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-278fef69-832b-4f27-8b63-d48bb2561ccd admin admin] The token cache is being invalidate because project d072351d1e014772891772d4a6989d92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:17:15.723953 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-278fef69-832b-4f27-8b63-d48bb2561ccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:15.724621 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-278fef69-832b-4f27-8b63-d48bb2561ccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:17:15.726244 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1413/2810] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:17:15 2022] DELETE /identity/v3/projects/d072351d1e014772891772d4a6989d92 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:15.732267 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb31a0-fe72-4305-ac3f-3285f8fd59da None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:15.732771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb31a0-fe72-4305-ac3f-3285f8fd59da None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:15.732997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb31a0-fe72-4305-ac3f-3285f8fd59da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:15.753939 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-17fb31a0-fe72-4305-ac3f-3285f8fd59da None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:15.755172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-17fb31a0-fe72-4305-ac3f-3285f8fd59da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:15.817959 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1398/2811] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:15.826266 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5b11d-1cd0-44d3-b7a4-b6f957adbaa5 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:15.826578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5b11d-1cd0-44d3-b7a4-b6f957adbaa5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:15.826852 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5b11d-1cd0-44d3-b7a4-b6f957adbaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:15.848703 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d9e5b11d-1cd0-44d3-b7a4-b6f957adbaa5 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:15.850202 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d9e5b11d-1cd0-44d3-b7a4-b6f957adbaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:15.858495 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1414/2812] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:15.863298 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb31a0-fe72-4305-ac3f-3285f8fd59da None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:15.873775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb31a0-fe72-4305-ac3f-3285f8fd59da None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:15.878497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab9ae0-a6fb-422e-bfe0-e1a0d56f70ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:15.878654 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab9ae0-a6fb-422e-bfe0-e1a0d56f70ce admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:15.878904 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab9ae0-a6fb-422e-bfe0-e1a0d56f70ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:15.879198 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab9ae0-a6fb-422e-bfe0-e1a0d56f70ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:15.879871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab9ae0-a6fb-422e-bfe0-e1a0d56f70ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:17:15.881952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab9ae0-a6fb-422e-bfe0-e1a0d56f70ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:15.881952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab9ae0-a6fb-422e-bfe0-e1a0d56f70ce admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:15.893718 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1399/2813] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:17:15.898314 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5b11d-1cd0-44d3-b7a4-b6f957adbaa5 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:15.908820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5b11d-1cd0-44d3-b7a4-b6f957adbaa5 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:15.913654 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1a6c9-3f16-4b5c-b914-b5cc0a1f97ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:15.914367 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1a6c9-3f16-4b5c-b914-b5cc0a1f97ed admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:15.914731 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1a6c9-3f16-4b5c-b914-b5cc0a1f97ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:15.915063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1a6c9-3f16-4b5c-b914-b5cc0a1f97ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:15.915701 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1a6c9-3f16-4b5c-b914-b5cc0a1f97ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:15.917597 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1a6c9-3f16-4b5c-b914-b5cc0a1f97ed admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:15.950143 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1415/2814] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:17:15 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:17:15.951437 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab9ae0-a6fb-422e-bfe0-e1a0d56f70ce admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:15.962676 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab9ae0-a6fb-422e-bfe0-e1a0d56f70ce admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:16.024425 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8831465-0ac0-4122-bdd2-9c037440880e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:16.025039 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8831465-0ac0-4122-bdd2-9c037440880e admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:16.025317 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8831465-0ac0-4122-bdd2-9c037440880e admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:16.025589 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8831465-0ac0-4122-bdd2-9c037440880e admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:16.026247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8831465-0ac0-4122-bdd2-9c037440880e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:16.028163 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8831465-0ac0-4122-bdd2-9c037440880e admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:16.039280 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:16.056020 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1400/2815] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:17:15 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:17:16.057475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1a6c9-3f16-4b5c-b914-b5cc0a1f97ed admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:16.068327 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1a6c9-3f16-4b5c-b914-b5cc0a1f97ed admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:16.073263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681d6db-c207-4010-8e74-c5c586d917d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:16.073878 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681d6db-c207-4010-8e74-c5c586d917d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:16.074190 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681d6db-c207-4010-8e74-c5c586d917d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:16.074468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681d6db-c207-4010-8e74-c5c586d917d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:16.075115 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681d6db-c207-4010-8e74-c5c586d917d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:16.076708 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681d6db-c207-4010-8e74-c5c586d917d9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:16.088336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1416/2816] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:17:16.092581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8831465-0ac0-4122-bdd2-9c037440880e admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:16.104386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8831465-0ac0-4122-bdd2-9c037440880e admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:16.109160 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acee978-df55-49fc-8600-d9862b099fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:16.109772 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acee978-df55-49fc-8600-d9862b099fdf admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:16.110046 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acee978-df55-49fc-8600-d9862b099fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:16.110958 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acee978-df55-49fc-8600-d9862b099fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:16.111491 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acee978-df55-49fc-8600-d9862b099fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:16.113107 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acee978-df55-49fc-8600-d9862b099fdf admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:16.124969 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1401/2817] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:17:16.129780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681d6db-c207-4010-8e74-c5c586d917d9 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:16.139992 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681d6db-c207-4010-8e74-c5c586d917d9 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:16.147452 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a161fb84-407c-4d9a-9307-064211faf81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:16.148117 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161fb84-407c-4d9a-9307-064211faf81f admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:16.148393 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161fb84-407c-4d9a-9307-064211faf81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:16.148663 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161fb84-407c-4d9a-9307-064211faf81f admin admin] PATH_INFO: `/v3/projects/feebd5cb20b144498610413f3a7dd94d/users/060ecaaa95a647239e3a20525df65980/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:16.164203 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a161fb84-407c-4d9a-9307-064211faf81f admin admin] RBAC: Authorizing `identity:create_grant(project_id=feebd5cb20b144498610413f3a7dd94d, user_id=060ecaaa95a647239e3a20525df65980, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:16.167001 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a161fb84-407c-4d9a-9307-064211faf81f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:16.181705 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1417/2818] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:17:16 2022] PUT /identity/v3/projects/feebd5cb20b144498610413f3a7dd94d/users/060ecaaa95a647239e3a20525df65980/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:17:16.193824 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d486f-314f-44f4-a77d-a9593ccb5f0a None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:16.194103 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d486f-314f-44f4-a77d-a9593ccb5f0a None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:16.194408 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d486f-314f-44f4-a77d-a9593ccb5f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:16.222226 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-431d486f-314f-44f4-a77d-a9593ccb5f0a None None] MFA Rules not processed for user `060ecaaa95a647239e3a20525df65980`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:16.223453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-431d486f-314f-44f4-a77d-a9593ccb5f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:16.292063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1402/2819] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:17:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:16.305322 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a161fb84-407c-4d9a-9307-064211faf81f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:16.307176 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a161fb84-407c-4d9a-9307-064211faf81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:16.386523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a161fb84-407c-4d9a-9307-064211faf81f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:16.390737 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27706d5-6b21-4bba-bda7-25c61d3b51f1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d477a49a1724b7e8a4699a5ebd04bde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:16.391418 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27706d5-6b21-4bba-bda7-25c61d3b51f1 service glance] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:16.392454 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27706d5-6b21-4bba-bda7-25c61d3b51f1 service glance] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:16.392843 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27706d5-6b21-4bba-bda7-25c61d3b51f1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:16.402144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27706d5-6b21-4bba-bda7-25c61d3b51f1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:16.403686 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27706d5-6b21-4bba-bda7-25c61d3b51f1 service glance] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:16.434948 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1418/2820] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:17:26.288582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:INSERT=1,keystone:SELECT=39 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:26.418035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:06.632424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d486f-314f-44f4-a77d-a9593ccb5f0a None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:06.633948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-431d486f-314f-44f4-a77d-a9593ccb5f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:06.699630 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d486f-314f-44f4-a77d-a9593ccb5f0a None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:06.705497 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6101a-0ff4-492d-adf4-988609eb65e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:06.706322 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6101a-0ff4-492d-adf4-988609eb65e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:06.706647 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6101a-0ff4-492d-adf4-988609eb65e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:06.706872 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6101a-0ff4-492d-adf4-988609eb65e8 admin admin] PATH_INFO: `/v3/users/af68b6ae8aea45b6aa7577d227ac348b` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:06.708327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6101a-0ff4-492d-adf4-988609eb65e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af68b6ae8aea45b6aa7577d227ac348b)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:06.710295 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6101a-0ff4-492d-adf4-988609eb65e8 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:06.769285 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5cd6101a-0ff4-492d-adf4-988609eb65e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af68b6ae8aea45b6aa7577d227ac348b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:06.780137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5cd6101a-0ff4-492d-adf4-988609eb65e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af68b6ae8aea45b6aa7577d227ac348b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:06.795391 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5cd6101a-0ff4-492d-adf4-988609eb65e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af68b6ae8aea45b6aa7577d227ac348b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:06.803287 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5cd6101a-0ff4-492d-adf4-988609eb65e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af68b6ae8aea45b6aa7577d227ac348b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:06.804018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-5cd6101a-0ff4-492d-adf4-988609eb65e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af68b6ae8aea45b6aa7577d227ac348b'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:06.813532 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1403/2821] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:18:06 2022] DELETE /identity/v3/users/af68b6ae8aea45b6aa7577d227ac348b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:06.816100 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3da39c-db9a-4769-9f6c-0b3d1d01dd11 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:06.816634 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3da39c-db9a-4769-9f6c-0b3d1d01dd11 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:06.816634 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3da39c-db9a-4769-9f6c-0b3d1d01dd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:06.838248 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2b3da39c-db9a-4769-9f6c-0b3d1d01dd11 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:06.839555 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2b3da39c-db9a-4769-9f6c-0b3d1d01dd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:06.904818 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1419/2822] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:06.915303 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6101a-0ff4-492d-adf4-988609eb65e8 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:06.916699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-5cd6101a-0ff4-492d-adf4-988609eb65e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:06.972441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6101a-0ff4-492d-adf4-988609eb65e8 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:06.976582 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166afed-5194-472e-8e09-6039f9e6b160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:06.977171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166afed-5194-472e-8e09-6039f9e6b160 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:06.977444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166afed-5194-472e-8e09-6039f9e6b160 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:06.977708 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166afed-5194-472e-8e09-6039f9e6b160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:06.986572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166afed-5194-472e-8e09-6039f9e6b160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:06.988014 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166afed-5194-472e-8e09-6039f9e6b160 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:06.998380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1404/2823] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:07.299210 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3da39c-db9a-4769-9f6c-0b3d1d01dd11 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:07.302280 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2b3da39c-db9a-4769-9f6c-0b3d1d01dd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:07.320931 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3da39c-db9a-4769-9f6c-0b3d1d01dd11 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:07.325945 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c658c4d-af24-45d2-b453-e6f06ce5a2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:07.326686 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c658c4d-af24-45d2-b453-e6f06ce5a2fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:07.327030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c658c4d-af24-45d2-b453-e6f06ce5a2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:07.327477 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c658c4d-af24-45d2-b453-e6f06ce5a2fa admin admin] PATH_INFO: `/v3/projects/e8d5d8c07a5441d1bc053d9334d23b6e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:07.328991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c658c4d-af24-45d2-b453-e6f06ce5a2fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8d5d8c07a5441d1bc053d9334d23b6e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:07.331253 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c658c4d-af24-45d2-b453-e6f06ce5a2fa admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:07.383049 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0c658c4d-af24-45d2-b453-e6f06ce5a2fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8d5d8c07a5441d1bc053d9334d23b6e', 'request_id': 'req-0c658c4d-af24-45d2-b453-e6f06ce5a2fa'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:07.390249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0c658c4d-af24-45d2-b453-e6f06ce5a2fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8d5d8c07a5441d1bc053d9334d23b6e', 'request_id': 'req-0c658c4d-af24-45d2-b453-e6f06ce5a2fa'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:07.396896 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0c658c4d-af24-45d2-b453-e6f06ce5a2fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8d5d8c07a5441d1bc053d9334d23b6e', 'request_id': 'req-0c658c4d-af24-45d2-b453-e6f06ce5a2fa'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:07.409260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0c658c4d-af24-45d2-b453-e6f06ce5a2fa admin admin] The token cache is being invalidate because project e8d5d8c07a5441d1bc053d9334d23b6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:18:07.409552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0c658c4d-af24-45d2-b453-e6f06ce5a2fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:07.410286 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0c658c4d-af24-45d2-b453-e6f06ce5a2fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:07.412676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1420/2824] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:18:07 2022] DELETE /identity/v3/projects/e8d5d8c07a5441d1bc053d9334d23b6e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:07.417855 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf3391-5853-4fd5-8522-f5c05bfaa2b4 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:07.418167 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf3391-5853-4fd5-8522-f5c05bfaa2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:07.418440 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf3391-5853-4fd5-8522-f5c05bfaa2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:07.441268 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-3ebf3391-5853-4fd5-8522-f5c05bfaa2b4 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:07.442560 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-3ebf3391-5853-4fd5-8522-f5c05bfaa2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:07.463028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:07.515745 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1405/2825] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:18:07.579954 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be257127-ba87-4d50-8ae5-3c740ec23d9a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:07.580241 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be257127-ba87-4d50-8ae5-3c740ec23d9a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:07.580747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be257127-ba87-4d50-8ae5-3c740ec23d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:07.602011 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-be257127-ba87-4d50-8ae5-3c740ec23d9a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:07.603728 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-be257127-ba87-4d50-8ae5-3c740ec23d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:07.614002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1421/2826] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:07.618490 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf3391-5853-4fd5-8522-f5c05bfaa2b4 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:07.628269 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf3391-5853-4fd5-8522-f5c05bfaa2b4 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:07.633519 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d1e0d-7abd-4e81-9ec8-96349e896bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:07.634102 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d1e0d-7abd-4e81-9ec8-96349e896bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:07.634414 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d1e0d-7abd-4e81-9ec8-96349e896bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:07.634689 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d1e0d-7abd-4e81-9ec8-96349e896bb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:07.635339 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d1e0d-7abd-4e81-9ec8-96349e896bb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:18:07.635638 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d1e0d-7abd-4e81-9ec8-96349e896bb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:07.637227 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d1e0d-7abd-4e81-9ec8-96349e896bb8 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:07.649971 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1406/2827] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:18:07.654467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be257127-ba87-4d50-8ae5-3c740ec23d9a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:07.663556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be257127-ba87-4d50-8ae5-3c740ec23d9a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:07.668260 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e56dd-c94a-4d46-a557-a30edea48e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:07.668858 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e56dd-c94a-4d46-a557-a30edea48e25 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:07.669135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e56dd-c94a-4d46-a557-a30edea48e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:07.669407 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e56dd-c94a-4d46-a557-a30edea48e25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:07.670044 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e56dd-c94a-4d46-a557-a30edea48e25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:07.671963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e56dd-c94a-4d46-a557-a30edea48e25 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:07.700644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1422/2828] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:18:07 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:18:07.705013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d1e0d-7abd-4e81-9ec8-96349e896bb8 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:07.715552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d1e0d-7abd-4e81-9ec8-96349e896bb8 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:07.772420 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d342ef5-9fbf-4738-b1cf-93a9398dd499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:07.773034 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d342ef5-9fbf-4738-b1cf-93a9398dd499 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:07.773320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d342ef5-9fbf-4738-b1cf-93a9398dd499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:07.773588 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d342ef5-9fbf-4738-b1cf-93a9398dd499 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:07.774248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d342ef5-9fbf-4738-b1cf-93a9398dd499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:07.776151 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d342ef5-9fbf-4738-b1cf-93a9398dd499 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:07.796526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1407/2829] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:18:07 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:07.800781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e56dd-c94a-4d46-a557-a30edea48e25 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:07.812439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e56dd-c94a-4d46-a557-a30edea48e25 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:07.817634 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217c418-e884-41b9-ba97-e9d98d4c5fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:07.818265 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217c418-e884-41b9-ba97-e9d98d4c5fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:07.818513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217c418-e884-41b9-ba97-e9d98d4c5fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:07.818778 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217c418-e884-41b9-ba97-e9d98d4c5fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:07.819414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217c418-e884-41b9-ba97-e9d98d4c5fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:07.820984 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217c418-e884-41b9-ba97-e9d98d4c5fa0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:07.831063 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1423/2830] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:07.835652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d342ef5-9fbf-4738-b1cf-93a9398dd499 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:07.845816 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d342ef5-9fbf-4738-b1cf-93a9398dd499 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:07.850361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46671b3d-cf04-4edd-8308-195c5aea46c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:07.850944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46671b3d-cf04-4edd-8308-195c5aea46c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:07.851216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46671b3d-cf04-4edd-8308-195c5aea46c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:07.851482 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46671b3d-cf04-4edd-8308-195c5aea46c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:07.852111 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46671b3d-cf04-4edd-8308-195c5aea46c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:07.853744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46671b3d-cf04-4edd-8308-195c5aea46c0 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:07.864147 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1408/2831] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:07.868284 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217c418-e884-41b9-ba97-e9d98d4c5fa0 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:07.878515 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217c418-e884-41b9-ba97-e9d98d4c5fa0 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:07.884074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548b2fa-69fb-4f64-9c14-6abada2b9835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:07.884751 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548b2fa-69fb-4f64-9c14-6abada2b9835 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:07.885511 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548b2fa-69fb-4f64-9c14-6abada2b9835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:07.885511 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548b2fa-69fb-4f64-9c14-6abada2b9835 admin admin] PATH_INFO: `/v3/projects/26d88da533a4432a97eb9644ce21fcfe/users/7e4d42a3879f4ec2b422f4bc8dd2dea9/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:07.901481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548b2fa-69fb-4f64-9c14-6abada2b9835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d88da533a4432a97eb9644ce21fcfe, user_id=7e4d42a3879f4ec2b422f4bc8dd2dea9, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:07.904280 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548b2fa-69fb-4f64-9c14-6abada2b9835 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:07.906078 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46671b3d-cf04-4edd-8308-195c5aea46c0 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:07.907831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-46671b3d-cf04-4edd-8308-195c5aea46c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:07.918690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1424/2832] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:07 2022] PUT /identity/v3/projects/26d88da533a4432a97eb9644ce21fcfe/users/7e4d42a3879f4ec2b422f4bc8dd2dea9/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:07.923482 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738dae5-3509-4248-8b30-ebda0c51e4a8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:07.923771 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738dae5-3509-4248-8b30-ebda0c51e4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:07.924049 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738dae5-3509-4248-8b30-ebda0c51e4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:07.924260 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46671b3d-cf04-4edd-8308-195c5aea46c0 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:07.944086 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e738dae5-3509-4248-8b30-ebda0c51e4a8 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:07.945363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e738dae5-3509-4248-8b30-ebda0c51e4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:07.963901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:07.984804 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd02a7-d515-4144-88ac-bb2d6011f6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:07.985447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd02a7-d515-4144-88ac-bb2d6011f6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:07.985728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd02a7-d515-4144-88ac-bb2d6011f6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:07.986055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd02a7-d515-4144-88ac-bb2d6011f6b9 admin admin] PATH_INFO: `/v3/users/060ecaaa95a647239e3a20525df65980` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:07.987415 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd02a7-d515-4144-88ac-bb2d6011f6b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=060ecaaa95a647239e3a20525df65980)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:07.989416 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd02a7-d515-4144-88ac-bb2d6011f6b9 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:08.017220 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1425/2833] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:18:08.024151 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:08.027993 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738dae5-3509-4248-8b30-ebda0c51e4a8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:08.032856 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e738dae5-3509-4248-8b30-ebda0c51e4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:08.040122 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-29dd02a7-d515-4144-88ac-bb2d6011f6b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '060ecaaa95a647239e3a20525df65980'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:08.048730 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-29dd02a7-d515-4144-88ac-bb2d6011f6b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '060ecaaa95a647239e3a20525df65980'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:08.064379 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-29dd02a7-d515-4144-88ac-bb2d6011f6b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '060ecaaa95a647239e3a20525df65980'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:08.071987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-29dd02a7-d515-4144-88ac-bb2d6011f6b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '060ecaaa95a647239e3a20525df65980'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:08.073175 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-29dd02a7-d515-4144-88ac-bb2d6011f6b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '060ecaaa95a647239e3a20525df65980'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:08.080001 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1409/2834] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:18:07 2022] DELETE /identity/v3/users/060ecaaa95a647239e3a20525df65980 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:18:08.084356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8742b-3c84-43fc-bd12-fc2beee8b704 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:08.084584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8742b-3c84-43fc-bd12-fc2beee8b704 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:08.084862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8742b-3c84-43fc-bd12-fc2beee8b704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:08.101733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738dae5-3509-4248-8b30-ebda0c51e4a8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:08.104703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-39e8742b-3c84-43fc-bd12-fc2beee8b704 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:08.105925 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-39e8742b-3c84-43fc-bd12-fc2beee8b704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:08.154332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba0d6d-f4d5-496a-a200-bc412df3df75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:08.154939 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba0d6d-f4d5-496a-a200-bc412df3df75 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:08.155211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba0d6d-f4d5-496a-a200-bc412df3df75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:08.155475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba0d6d-f4d5-496a-a200-bc412df3df75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:08.157254 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5dba0d6d-f4d5-496a-a200-bc412df3df75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:08.167973 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1410/2835] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:18:08.178436 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8742b-3c84-43fc-bd12-fc2beee8b704 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:08.179832 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-39e8742b-3c84-43fc-bd12-fc2beee8b704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:08.191751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8742b-3c84-43fc-bd12-fc2beee8b704 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:08.200413 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d564c6-ac38-45a9-8d34-4ee3564f61cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:08.201044 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d564c6-ac38-45a9-8d34-4ee3564f61cf service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:08.201327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d564c6-ac38-45a9-8d34-4ee3564f61cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:08.201598 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d564c6-ac38-45a9-8d34-4ee3564f61cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:08.212876 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d564c6-ac38-45a9-8d34-4ee3564f61cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:08.214361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d564c6-ac38-45a9-8d34-4ee3564f61cf service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:08.227865 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1411/2836] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:18:08.228178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba0d6d-f4d5-496a-a200-bc412df3df75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:08.229793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba0d6d-f4d5-496a-a200-bc412df3df75 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:08.244702 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1426/2837] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:18:08.519285 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d564c6-ac38-45a9-8d34-4ee3564f61cf service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:08.520831 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-18d564c6-ac38-45a9-8d34-4ee3564f61cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:08.547677 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d564c6-ac38-45a9-8d34-4ee3564f61cf service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:08.563118 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427b5ed-3671-4fd4-95c6-e6765e219ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:08.563906 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427b5ed-3671-4fd4-95c6-e6765e219ee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:08.564327 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427b5ed-3671-4fd4-95c6-e6765e219ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:08.564564 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427b5ed-3671-4fd4-95c6-e6765e219ee4 admin admin] PATH_INFO: `/v3/projects/feebd5cb20b144498610413f3a7dd94d` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:08.566264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427b5ed-3671-4fd4-95c6-e6765e219ee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=feebd5cb20b144498610413f3a7dd94d)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:08.568474 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427b5ed-3671-4fd4-95c6-e6765e219ee4 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:08.633710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c427b5ed-3671-4fd4-95c6-e6765e219ee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feebd5cb20b144498610413f3a7dd94d', 'request_id': 'req-c427b5ed-3671-4fd4-95c6-e6765e219ee4'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:08.642096 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c427b5ed-3671-4fd4-95c6-e6765e219ee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feebd5cb20b144498610413f3a7dd94d', 'request_id': 'req-c427b5ed-3671-4fd4-95c6-e6765e219ee4'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:08.650151 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c427b5ed-3671-4fd4-95c6-e6765e219ee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feebd5cb20b144498610413f3a7dd94d', 'request_id': 'req-c427b5ed-3671-4fd4-95c6-e6765e219ee4'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:08.664065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c427b5ed-3671-4fd4-95c6-e6765e219ee4 admin admin] The token cache is being invalidate because project feebd5cb20b144498610413f3a7dd94d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:18:08.664362 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c427b5ed-3671-4fd4-95c6-e6765e219ee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:08.666551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-c427b5ed-3671-4fd4-95c6-e6765e219ee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:08.673118 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1412/2838] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:18:08 2022] DELETE /identity/v3/projects/feebd5cb20b144498610413f3a7dd94d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:08.674986 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08f724-62e2-4b4f-bc84-d26e15d55020 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:08.675434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08f724-62e2-4b4f-bc84-d26e15d55020 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:08.675642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08f724-62e2-4b4f-bc84-d26e15d55020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:08.698833 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-8d08f724-62e2-4b4f-bc84-d26e15d55020 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:08.700057 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8d08f724-62e2-4b4f-bc84-d26e15d55020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:08.721427 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:08.765637 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1427/2839] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:08.774286 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a5bb0-2760-4e3e-b85d-e38988c13c13 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:08.774910 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a5bb0-2760-4e3e-b85d-e38988c13c13 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:08.774910 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a5bb0-2760-4e3e-b85d-e38988c13c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:08.795383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-5d9a5bb0-2760-4e3e-b85d-e38988c13c13 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:08.796618 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-5d9a5bb0-2760-4e3e-b85d-e38988c13c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:08.809576 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1413/2840] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:08.813821 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08f724-62e2-4b4f-bc84-d26e15d55020 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:08.823961 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08f724-62e2-4b4f-bc84-d26e15d55020 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:08.828829 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d1416-8d8f-49ef-8902-4a6e68a3246b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:08.829440 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d1416-8d8f-49ef-8902-4a6e68a3246b admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:08.829719 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d1416-8d8f-49ef-8902-4a6e68a3246b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:08.829996 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d1416-8d8f-49ef-8902-4a6e68a3246b admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:08.830681 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d1416-8d8f-49ef-8902-4a6e68a3246b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:18:08.830987 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d1416-8d8f-49ef-8902-4a6e68a3246b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:08.833661 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d1416-8d8f-49ef-8902-4a6e68a3246b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:08.845930 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1428/2841] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:18:08.851870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a5bb0-2760-4e3e-b85d-e38988c13c13 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:08.876443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a5bb0-2760-4e3e-b85d-e38988c13c13 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:08.881758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ecd10-a703-45a5-8430-fefcc0ad8fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:08.882670 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ecd10-a703-45a5-8430-fefcc0ad8fb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:08.882955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ecd10-a703-45a5-8430-fefcc0ad8fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:08.883225 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ecd10-a703-45a5-8430-fefcc0ad8fb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:08.883867 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ecd10-a703-45a5-8430-fefcc0ad8fb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:08.885800 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ecd10-a703-45a5-8430-fefcc0ad8fb8 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:08.899396 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:08.925320 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1414/2842] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:18:08 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:08.929647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d1416-8d8f-49ef-8902-4a6e68a3246b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:08.941733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d1416-8d8f-49ef-8902-4a6e68a3246b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:09.013234 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8c8e9-8534-4a9f-b99e-8d070cba1212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:09.013974 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8c8e9-8534-4a9f-b99e-8d070cba1212 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:09.014320 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8c8e9-8534-4a9f-b99e-8d070cba1212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:09.014632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8c8e9-8534-4a9f-b99e-8d070cba1212 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:09.015359 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8c8e9-8534-4a9f-b99e-8d070cba1212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:09.019099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8c8e9-8534-4a9f-b99e-8d070cba1212 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:09.044513 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1429/2843] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:18:08 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:09.049084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ecd10-a703-45a5-8430-fefcc0ad8fb8 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:09.060803 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ecd10-a703-45a5-8430-fefcc0ad8fb8 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:09.065634 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e716cab-b352-4930-b2ac-f42dee73c73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:09.066325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e716cab-b352-4930-b2ac-f42dee73c73b admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:09.066609 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e716cab-b352-4930-b2ac-f42dee73c73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:09.066932 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e716cab-b352-4930-b2ac-f42dee73c73b admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:09.067584 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e716cab-b352-4930-b2ac-f42dee73c73b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:09.069218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e716cab-b352-4930-b2ac-f42dee73c73b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:09.082173 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1415/2844] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:09.085630 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8c8e9-8534-4a9f-b99e-8d070cba1212 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:09.098733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8c8e9-8534-4a9f-b99e-8d070cba1212 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:09.106277 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d0943-1f12-4bc0-b57f-7d4b5f7ab1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:09.107214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d0943-1f12-4bc0-b57f-7d4b5f7ab1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:09.107691 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d0943-1f12-4bc0-b57f-7d4b5f7ab1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:09.108732 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d0943-1f12-4bc0-b57f-7d4b5f7ab1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:09.108732 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d0943-1f12-4bc0-b57f-7d4b5f7ab1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:09.110400 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d0943-1f12-4bc0-b57f-7d4b5f7ab1b0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:09.123724 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1430/2845] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:09.127757 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e716cab-b352-4930-b2ac-f42dee73c73b admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:09.139639 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e716cab-b352-4930-b2ac-f42dee73c73b admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:09.145026 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69593c98-d31c-41a8-957e-5a627f416514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:09.145769 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593c98-d31c-41a8-957e-5a627f416514 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:09.146069 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593c98-d31c-41a8-957e-5a627f416514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:09.146369 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593c98-d31c-41a8-957e-5a627f416514 admin admin] PATH_INFO: `/v3/projects/1490fa450e9c4b5c8089f538b9850c16/users/d94ce297605b497e877dd74c5c14140e/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:09.167155 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69593c98-d31c-41a8-957e-5a627f416514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1490fa450e9c4b5c8089f538b9850c16, user_id=d94ce297605b497e877dd74c5c14140e, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:09.170072 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69593c98-d31c-41a8-957e-5a627f416514 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:09.198047 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1416/2846] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:09 2022] PUT /identity/v3/projects/1490fa450e9c4b5c8089f538b9850c16/users/d94ce297605b497e877dd74c5c14140e/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:09.200501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834dedd-c0da-4381-bc95-72323322a516 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:09.200819 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834dedd-c0da-4381-bc95-72323322a516 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:09.201047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834dedd-c0da-4381-bc95-72323322a516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:09.238163 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2834dedd-c0da-4381-bc95-72323322a516 None None] MFA Rules not processed for user `d94ce297605b497e877dd74c5c14140e`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:09.240058 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2834dedd-c0da-4381-bc95-72323322a516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:09.313939 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1431/2847] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:18:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:09.326502 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69593c98-d31c-41a8-957e-5a627f416514 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:09.328262 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-69593c98-d31c-41a8-957e-5a627f416514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:09.395335 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69593c98-d31c-41a8-957e-5a627f416514 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:09.398844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cacfcea-f5fc-40fc-a10d-b9b2fa220d16 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d477a49a1724b7e8a4699a5ebd04bde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:09.399453 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacfcea-f5fc-40fc-a10d-b9b2fa220d16 service glance] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:09.399728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacfcea-f5fc-40fc-a10d-b9b2fa220d16 service glance] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:09.399992 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacfcea-f5fc-40fc-a10d-b9b2fa220d16 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:09.409376 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cacfcea-f5fc-40fc-a10d-b9b2fa220d16 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:09.411375 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cacfcea-f5fc-40fc-a10d-b9b2fa220d16 service glance] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:09.421117 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1417/2848] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:12.620377 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c655b-3823-4179-832e-3009094e3e99 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:12.621147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c655b-3823-4179-832e-3009094e3e99 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:12.621277 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c655b-3823-4179-832e-3009094e3e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:12.653409 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-3c1c655b-3823-4179-832e-3009094e3e99 None None] MFA Rules not processed for user `7e4d42a3879f4ec2b422f4bc8dd2dea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:12.654764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3c1c655b-3823-4179-832e-3009094e3e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:12.665175 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:12.740386 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1432/2849] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:18:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:12.744920 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cacfcea-f5fc-40fc-a10d-b9b2fa220d16 service glance] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:12.746744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6cacfcea-f5fc-40fc-a10d-b9b2fa220d16 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:12.818406 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cacfcea-f5fc-40fc-a10d-b9b2fa220d16 service glance] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:12.823888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc883a95-475a-49a8-989b-2b5ee03f81ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:12.824551 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc883a95-475a-49a8-989b-2b5ee03f81ce admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:12.824829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc883a95-475a-49a8-989b-2b5ee03f81ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:12.825108 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc883a95-475a-49a8-989b-2b5ee03f81ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:12.825896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc883a95-475a-49a8-989b-2b5ee03f81ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:12.827881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc883a95-475a-49a8-989b-2b5ee03f81ce admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:12.866448 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1418/2850] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:18:12 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:12.867774 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c655b-3823-4179-832e-3009094e3e99 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:12.885285 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c655b-3823-4179-832e-3009094e3e99 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:12.966569 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d3907-1600-433e-9cbc-eac9711b6874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:12.967205 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d3907-1600-433e-9cbc-eac9711b6874 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:12.967481 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d3907-1600-433e-9cbc-eac9711b6874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:12.967750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d3907-1600-433e-9cbc-eac9711b6874 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:12.968700 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d3907-1600-433e-9cbc-eac9711b6874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:12.971079 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d3907-1600-433e-9cbc-eac9711b6874 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:13.002336 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1433/2851] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:18:12 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:13.006583 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc883a95-475a-49a8-989b-2b5ee03f81ce admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:13.018183 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc883a95-475a-49a8-989b-2b5ee03f81ce admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:13.024077 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0acb3-29cf-4066-9488-671e46320f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:13.024698 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0acb3-29cf-4066-9488-671e46320f55 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:13.024979 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0acb3-29cf-4066-9488-671e46320f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:13.025281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0acb3-29cf-4066-9488-671e46320f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:13.025949 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0acb3-29cf-4066-9488-671e46320f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:13.027612 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0acb3-29cf-4066-9488-671e46320f55 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:13.039964 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1419/2852] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:18:13.044624 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d3907-1600-433e-9cbc-eac9711b6874 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:13.058002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d3907-1600-433e-9cbc-eac9711b6874 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:13.063401 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c99a4-de9c-4009-81a0-8cc245b464d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:13.064030 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c99a4-de9c-4009-81a0-8cc245b464d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:13.064517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c99a4-de9c-4009-81a0-8cc245b464d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:13.064658 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c99a4-de9c-4009-81a0-8cc245b464d0 admin admin] PATH_INFO: `/v3/projects/7b33b931648b49c198607d02eca7c406/users/6648cfb8e2fd4e0894f09223104df8e2/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:13.083220 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c99a4-de9c-4009-81a0-8cc245b464d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b33b931648b49c198607d02eca7c406, user_id=6648cfb8e2fd4e0894f09223104df8e2, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:13.085901 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c99a4-de9c-4009-81a0-8cc245b464d0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:13.110767 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1434/2853] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:13 2022] PUT /identity/v3/projects/7b33b931648b49c198607d02eca7c406/users/6648cfb8e2fd4e0894f09223104df8e2/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:18:13.114894 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0acb3-29cf-4066-9488-671e46320f55 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:13.126758 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0acb3-29cf-4066-9488-671e46320f55 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:13.161994 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:13.195203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b0cee-a952-4065-9be5-a69cb637e497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:13.196044 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b0cee-a952-4065-9be5-a69cb637e497 admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:13.196359 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b0cee-a952-4065-9be5-a69cb637e497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:13.196653 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b0cee-a952-4065-9be5-a69cb637e497 admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:13.197316 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b0cee-a952-4065-9be5-a69cb637e497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:13.198947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b0cee-a952-4065-9be5-a69cb637e497 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:13.219013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1420/2854] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:13.225714 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c99a4-de9c-4009-81a0-8cc245b464d0 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:13.236633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c99a4-de9c-4009-81a0-8cc245b464d0 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:13.242140 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dff43d-5d69-4e76-a3d6-eb728aa36e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:13.242807 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dff43d-5d69-4e76-a3d6-eb728aa36e26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:13.243084 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dff43d-5d69-4e76-a3d6-eb728aa36e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:13.243965 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dff43d-5d69-4e76-a3d6-eb728aa36e26 admin admin] PATH_INFO: `/v3/projects/7b33b931648b49c198607d02eca7c406/users/6648cfb8e2fd4e0894f09223104df8e2/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:13.245471 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dff43d-5d69-4e76-a3d6-eb728aa36e26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b33b931648b49c198607d02eca7c406, user_id=6648cfb8e2fd4e0894f09223104df8e2, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:13.248399 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dff43d-5d69-4e76-a3d6-eb728aa36e26 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:13.260297 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1435/2855] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:13 2022] PUT /identity/v3/projects/7b33b931648b49c198607d02eca7c406/users/6648cfb8e2fd4e0894f09223104df8e2/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:18:18.158104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318c3f3-34fe-4c3f-985a-1bd7abeb1874 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:18.158732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318c3f3-34fe-4c3f-985a-1bd7abeb1874 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:18.158732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318c3f3-34fe-4c3f-985a-1bd7abeb1874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:18.188907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-2318c3f3-34fe-4c3f-985a-1bd7abeb1874 None None] MFA Rules not processed for user `6648cfb8e2fd4e0894f09223104df8e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:18.189957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2318c3f3-34fe-4c3f-985a-1bd7abeb1874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:18.289703 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1421/2856] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:18:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:18.305844 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dff43d-5d69-4e76-a3d6-eb728aa36e26 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:18.308353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-88dff43d-5d69-4e76-a3d6-eb728aa36e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:18.395682 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:18.404047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dff43d-5d69-4e76-a3d6-eb728aa36e26 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:18.412010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6add2f-164f-4031-a243-748311ad6c68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:18.412644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6add2f-164f-4031-a243-748311ad6c68 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:18.412914 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6add2f-164f-4031-a243-748311ad6c68 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:18.413181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6add2f-164f-4031-a243-748311ad6c68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:18.474387 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6add2f-164f-4031-a243-748311ad6c68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:18.475977 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6add2f-164f-4031-a243-748311ad6c68 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:18.485779 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1436/2857] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:19.523085 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c38ed9-0f3c-4f4f-ab8e-03d5cf578662 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:19.523585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c38ed9-0f3c-4f4f-ab8e-03d5cf578662 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:19.523710 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c38ed9-0f3c-4f4f-ab8e-03d5cf578662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:19.550085 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-d4c38ed9-0f3c-4f4f-ab8e-03d5cf578662 None None] MFA Rules not processed for user `7e4d42a3879f4ec2b422f4bc8dd2dea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:19.552029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d4c38ed9-0f3c-4f4f-ab8e-03d5cf578662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:19.559018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1422/2858] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:18:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:28.481160 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=27 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:29.539933 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=52 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:51.255749 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6add2f-164f-4031-a243-748311ad6c68 service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:51.267606 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6add2f-164f-4031-a243-748311ad6c68 service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:51.274302 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e76cb-ba74-4af0-990f-c0c26c7805ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:51.275064 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e76cb-ba74-4af0-990f-c0c26c7805ca service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:51.275441 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e76cb-ba74-4af0-990f-c0c26c7805ca service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:51.275811 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e76cb-ba74-4af0-990f-c0c26c7805ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:51.277808 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cf2e76cb-ba74-4af0-990f-c0c26c7805ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:51.292468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e76cb-ba74-4af0-990f-c0c26c7805ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:51.294070 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e76cb-ba74-4af0-990f-c0c26c7805ca service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:51.306212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1437/2859] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:01.302316 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:02.584759 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c38ed9-0f3c-4f4f-ab8e-03d5cf578662 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:02.587069 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d4c38ed9-0f3c-4f4f-ab8e-03d5cf578662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:02.659799 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c38ed9-0f3c-4f4f-ab8e-03d5cf578662 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:02.664115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83684f0d-149e-4a85-9435-fe77a0ec02ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:02.664711 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83684f0d-149e-4a85-9435-fe77a0ec02ef service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:02.664986 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83684f0d-149e-4a85-9435-fe77a0ec02ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:02.665257 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83684f0d-149e-4a85-9435-fe77a0ec02ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:02.667016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-83684f0d-149e-4a85-9435-fe77a0ec02ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:02.676684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83684f0d-149e-4a85-9435-fe77a0ec02ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:02.678098 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83684f0d-149e-4a85-9435-fe77a0ec02ef service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:02.686937 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1423/2860] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:03.177552 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e76cb-ba74-4af0-990f-c0c26c7805ca service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:03.179753 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cf2e76cb-ba74-4af0-990f-c0c26c7805ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:03.259690 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e76cb-ba74-4af0-990f-c0c26c7805ca service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:03.266307 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8daba-4273-41ff-9483-a3b3c50d8c3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:03.266911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8daba-4273-41ff-9483-a3b3c50d8c3d service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:03.267178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8daba-4273-41ff-9483-a3b3c50d8c3d service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:03.267442 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8daba-4273-41ff-9483-a3b3c50d8c3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:03.269173 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-41b8daba-4273-41ff-9483-a3b3c50d8c3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:03.280326 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8daba-4273-41ff-9483-a3b3c50d8c3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:03.281873 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8daba-4273-41ff-9483-a3b3c50d8c3d service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:03.292279 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1438/2861] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:07.987485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee051a4-647b-440c-8cbe-e1c72ca678db None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:07.988401 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee051a4-647b-440c-8cbe-e1c72ca678db None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:07.988674 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee051a4-647b-440c-8cbe-e1c72ca678db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:08.013006 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-3ee051a4-647b-440c-8cbe-e1c72ca678db None None] MFA Rules not processed for user `f849563115ec4a4e859a7395a03a561a`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:08.014739 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-3ee051a4-647b-440c-8cbe-e1c72ca678db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:08.094170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1424/2862] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:19:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:10.293484 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8daba-4273-41ff-9483-a3b3c50d8c3d service placement] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:10.369676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8daba-4273-41ff-9483-a3b3c50d8c3d service placement] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:10.377363 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dff19-0960-4be4-8651-14aa32d259be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:10.378135 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dff19-0960-4be4-8651-14aa32d259be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:10.378520 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dff19-0960-4be4-8651-14aa32d259be admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:10.378891 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dff19-0960-4be4-8651-14aa32d259be admin admin] PATH_INFO: `/v3/users/d94ce297605b497e877dd74c5c14140e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:10.380290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960dff19-0960-4be4-8651-14aa32d259be admin admin] RBAC: Authorizing `identity:delete_user(user_id=d94ce297605b497e877dd74c5c14140e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:10.382509 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960dff19-0960-4be4-8651-14aa32d259be admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:10.435725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-960dff19-0960-4be4-8651-14aa32d259be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd94ce297605b497e877dd74c5c14140e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:10.439235 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-960dff19-0960-4be4-8651-14aa32d259be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd94ce297605b497e877dd74c5c14140e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:10.449249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-960dff19-0960-4be4-8651-14aa32d259be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd94ce297605b497e877dd74c5c14140e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:10.457290 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-960dff19-0960-4be4-8651-14aa32d259be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd94ce297605b497e877dd74c5c14140e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:10.470654 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-960dff19-0960-4be4-8651-14aa32d259be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd94ce297605b497e877dd74c5c14140e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:10.480492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1439/2863] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:10 2022] DELETE /identity/v3/users/d94ce297605b497e877dd74c5c14140e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:10.482928 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce3730-13a7-4dba-ab72-32c282f3831c None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:10.483838 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce3730-13a7-4dba-ab72-32c282f3831c None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:10.484351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce3730-13a7-4dba-ab72-32c282f3831c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:10.505955 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-44ce3730-13a7-4dba-ab72-32c282f3831c None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:10.507707 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-44ce3730-13a7-4dba-ab72-32c282f3831c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:10.583044 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1425/2864] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:10.596021 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dff19-0960-4be4-8651-14aa32d259be admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:10.598378 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-960dff19-0960-4be4-8651-14aa32d259be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:10.612545 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:10.683724 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dff19-0960-4be4-8651-14aa32d259be admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:10.688089 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5ed28-ab14-4038-90f6-f0831687d170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:10.688297 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5ed28-ab14-4038-90f6-f0831687d170 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:10.688529 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5ed28-ab14-4038-90f6-f0831687d170 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:10.689064 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5ed28-ab14-4038-90f6-f0831687d170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:10.700889 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5ed28-ab14-4038-90f6-f0831687d170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:10.702343 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5ed28-ab14-4038-90f6-f0831687d170 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:10.715037 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1440/2865] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:11.147617 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce3730-13a7-4dba-ab72-32c282f3831c None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:11.148957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-44ce3730-13a7-4dba-ab72-32c282f3831c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:11.160992 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce3730-13a7-4dba-ab72-32c282f3831c None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:11.175980 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b71b7-bd63-4737-9646-6de7243d0f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:11.176751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b71b7-bd63-4737-9646-6de7243d0f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:11.177140 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b71b7-bd63-4737-9646-6de7243d0f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:11.178156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b71b7-bd63-4737-9646-6de7243d0f05 admin admin] PATH_INFO: `/v3/projects/1490fa450e9c4b5c8089f538b9850c16` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:11.179944 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b71b7-bd63-4737-9646-6de7243d0f05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1490fa450e9c4b5c8089f538b9850c16)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:11.181943 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b71b7-bd63-4737-9646-6de7243d0f05 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:11.234384 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:11.241826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-662b71b7-bd63-4737-9646-6de7243d0f05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1490fa450e9c4b5c8089f538b9850c16', 'request_id': 'req-662b71b7-bd63-4737-9646-6de7243d0f05'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:11.255861 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-662b71b7-bd63-4737-9646-6de7243d0f05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1490fa450e9c4b5c8089f538b9850c16', 'request_id': 'req-662b71b7-bd63-4737-9646-6de7243d0f05'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:11.268134 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-662b71b7-bd63-4737-9646-6de7243d0f05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1490fa450e9c4b5c8089f538b9850c16', 'request_id': 'req-662b71b7-bd63-4737-9646-6de7243d0f05'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:11.288684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-662b71b7-bd63-4737-9646-6de7243d0f05 admin admin] The token cache is being invalidate because project 1490fa450e9c4b5c8089f538b9850c16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81447) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:11.289099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-662b71b7-bd63-4737-9646-6de7243d0f05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:11.290097 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-662b71b7-bd63-4737-9646-6de7243d0f05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:11.292264 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1426/2866] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:11 2022] DELETE /identity/v3/projects/1490fa450e9c4b5c8089f538b9850c16 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:11.296696 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac98c8d-b3c8-4d2d-b63c-8d6499c1cc16 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:11.297181 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac98c8d-b3c8-4d2d-b63c-8d6499c1cc16 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:11.297366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac98c8d-b3c8-4d2d-b63c-8d6499c1cc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:11.318661 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-0ac98c8d-b3c8-4d2d-b63c-8d6499c1cc16 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:11.320025 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0ac98c8d-b3c8-4d2d-b63c-8d6499c1cc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:11.384593 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1441/2867] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:11.394561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af03ee-7775-4d02-bc0c-f241ccadd138 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:11.394844 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af03ee-7775-4d02-bc0c-f241ccadd138 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:11.395113 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af03ee-7775-4d02-bc0c-f241ccadd138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:11.415055 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f2af03ee-7775-4d02-bc0c-f241ccadd138 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:11.416241 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f2af03ee-7775-4d02-bc0c-f241ccadd138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:11.424069 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1427/2868] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:11.428347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac98c8d-b3c8-4d2d-b63c-8d6499c1cc16 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:11.438755 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac98c8d-b3c8-4d2d-b63c-8d6499c1cc16 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:11.443475 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4d49b-fb48-4dc2-a99c-ed7b8a2f1614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:11.444109 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4d49b-fb48-4dc2-a99c-ed7b8a2f1614 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:11.444414 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4d49b-fb48-4dc2-a99c-ed7b8a2f1614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:11.444684 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4d49b-fb48-4dc2-a99c-ed7b8a2f1614 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:11.445326 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4d49b-fb48-4dc2-a99c-ed7b8a2f1614 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:19:11.445629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4d49b-fb48-4dc2-a99c-ed7b8a2f1614 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:11.447249 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4d49b-fb48-4dc2-a99c-ed7b8a2f1614 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:11.459352 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1442/2869] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:19:11.463561 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af03ee-7775-4d02-bc0c-f241ccadd138 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:11.472653 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af03ee-7775-4d02-bc0c-f241ccadd138 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:11.477963 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066d759-0d79-4f59-8764-8f58e059e10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:11.478585 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066d759-0d79-4f59-8764-8f58e059e10f admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:11.478862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066d759-0d79-4f59-8764-8f58e059e10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:11.479129 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066d759-0d79-4f59-8764-8f58e059e10f admin admin] PATH_INFO: `/v3/projects` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:11.479762 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066d759-0d79-4f59-8764-8f58e059e10f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:11.481652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066d759-0d79-4f59-8764-8f58e059e10f admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:11.510288 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1428/2870] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:11 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:11.514492 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4d49b-fb48-4dc2-a99c-ed7b8a2f1614 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:11.525038 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4d49b-fb48-4dc2-a99c-ed7b8a2f1614 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:11.583006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3970dcf-3ac8-474f-9ecd-f283070b8dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:11.583644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3970dcf-3ac8-474f-9ecd-f283070b8dfe admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:11.583920 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3970dcf-3ac8-474f-9ecd-f283070b8dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:11.584198 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3970dcf-3ac8-474f-9ecd-f283070b8dfe admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:11.584894 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3970dcf-3ac8-474f-9ecd-f283070b8dfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:11.586898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3970dcf-3ac8-474f-9ecd-f283070b8dfe admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:11.609903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1443/2871] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:11 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:11.614160 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066d759-0d79-4f59-8764-8f58e059e10f admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:11.625898 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066d759-0d79-4f59-8764-8f58e059e10f admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:11.630724 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d788789-06fa-494c-a7c4-a5eab8d0556b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:11.631419 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d788789-06fa-494c-a7c4-a5eab8d0556b admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:11.631814 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d788789-06fa-494c-a7c4-a5eab8d0556b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:11.632177 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d788789-06fa-494c-a7c4-a5eab8d0556b admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:11.632896 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d788789-06fa-494c-a7c4-a5eab8d0556b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:11.634631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d788789-06fa-494c-a7c4-a5eab8d0556b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:11.645142 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1429/2872] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:11.649089 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3970dcf-3ac8-474f-9ecd-f283070b8dfe admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:11.659711 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3970dcf-3ac8-474f-9ecd-f283070b8dfe admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:11.665098 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f468eb6-1a0e-492f-9f21-87a7bc6ad3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:11.665708 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f468eb6-1a0e-492f-9f21-87a7bc6ad3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:11.666027 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f468eb6-1a0e-492f-9f21-87a7bc6ad3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:11.666332 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f468eb6-1a0e-492f-9f21-87a7bc6ad3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:11.666981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f468eb6-1a0e-492f-9f21-87a7bc6ad3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:11.668604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f468eb6-1a0e-492f-9f21-87a7bc6ad3f3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:11.680185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1444/2873] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:11.684627 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d788789-06fa-494c-a7c4-a5eab8d0556b admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:11.695534 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d788789-06fa-494c-a7c4-a5eab8d0556b admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:11.700572 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfaecea-b82e-4a20-a23b-d88876641738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:11.701306 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaecea-b82e-4a20-a23b-d88876641738 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:11.701657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaecea-b82e-4a20-a23b-d88876641738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:11.702454 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaecea-b82e-4a20-a23b-d88876641738 admin admin] PATH_INFO: `/v3/projects/781d68e5617d4fcd838836192767453e/users/392ffc42301e4752beb3046b1ffca33f/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:11.719181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfaecea-b82e-4a20-a23b-d88876641738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=781d68e5617d4fcd838836192767453e, user_id=392ffc42301e4752beb3046b1ffca33f, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:11.721755 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfaecea-b82e-4a20-a23b-d88876641738 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:11.736502 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1430/2874] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:11 2022] PUT /identity/v3/projects/781d68e5617d4fcd838836192767453e/users/392ffc42301e4752beb3046b1ffca33f/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:11.749445 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81489c6-ab8e-438c-b6b8-f3856986e41f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:11.749764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81489c6-ab8e-438c-b6b8-f3856986e41f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:11.750146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81489c6-ab8e-438c-b6b8-f3856986e41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:11.782211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f81489c6-ab8e-438c-b6b8-f3856986e41f None None] MFA Rules not processed for user `392ffc42301e4752beb3046b1ffca33f`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:11.783684 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f81489c6-ab8e-438c-b6b8-f3856986e41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:11.788793 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:11.857214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1445/2875] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:19:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:11.868548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfaecea-b82e-4a20-a23b-d88876641738 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:11.870369 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-dbfaecea-b82e-4a20-a23b-d88876641738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:11.936910 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfaecea-b82e-4a20-a23b-d88876641738 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:11.940875 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a779cf-548a-4252-b415-301501f1f5e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:11.941500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a779cf-548a-4252-b415-301501f1f5e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:11.941779 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a779cf-548a-4252-b415-301501f1f5e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:11.942053 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a779cf-548a-4252-b415-301501f1f5e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:11.951838 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a779cf-548a-4252-b415-301501f1f5e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:11.953279 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a779cf-548a-4252-b415-301501f1f5e8 service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:11.963064 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1431/2876] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:12.237541 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81489c6-ab8e-438c-b6b8-f3856986e41f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:12.239382 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f81489c6-ab8e-438c-b6b8-f3856986e41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:12.373644 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81489c6-ab8e-438c-b6b8-f3856986e41f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:12.379858 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ebec4-73d1-4d56-ba39-ae7d4ecf1779 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:12.380490 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ebec4-73d1-4d56-ba39-ae7d4ecf1779 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:12.380824 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ebec4-73d1-4d56-ba39-ae7d4ecf1779 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:12.381023 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ebec4-73d1-4d56-ba39-ae7d4ecf1779 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:12.382908 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-401ebec4-73d1-4d56-ba39-ae7d4ecf1779 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:12.397544 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ebec4-73d1-4d56-ba39-ae7d4ecf1779 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:12.399781 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ebec4-73d1-4d56-ba39-ae7d4ecf1779 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:12.411474 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1446/2877] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:13.372449 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db893573-c656-4aa8-98bc-59f47905b7c0 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:13.373016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db893573-c656-4aa8-98bc-59f47905b7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:13.373016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db893573-c656-4aa8-98bc-59f47905b7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:13.392557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-db893573-c656-4aa8-98bc-59f47905b7c0 None None] MFA Rules not processed for user `392ffc42301e4752beb3046b1ffca33f`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:13.393751 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-db893573-c656-4aa8-98bc-59f47905b7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:13.400567 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1432/2878] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:19:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:18.504015 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ebec4-73d1-4d56-ba39-ae7d4ecf1779 service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:18.505709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-401ebec4-73d1-4d56-ba39-ae7d4ecf1779 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:18.522699 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ebec4-73d1-4d56-ba39-ae7d4ecf1779 service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:18.528353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb3132-86c9-4b66-b2b2-2eb4138338d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:18.528943 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb3132-86c9-4b66-b2b2-2eb4138338d8 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:18.529223 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb3132-86c9-4b66-b2b2-2eb4138338d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:18.529495 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb3132-86c9-4b66-b2b2-2eb4138338d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:18.531295 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cdeb3132-86c9-4b66-b2b2-2eb4138338d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:18.545234 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb3132-86c9-4b66-b2b2-2eb4138338d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:18.546849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb3132-86c9-4b66-b2b2-2eb4138338d8 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:18.559047 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1447/2879] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:23.387341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=64,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:28.554468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=64 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:30.624358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db893573-c656-4aa8-98bc-59f47905b7c0 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:30.626997 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-db893573-c656-4aa8-98bc-59f47905b7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:30.697352 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db893573-c656-4aa8-98bc-59f47905b7c0 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:30.700887 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bff58-2f6b-4af5-88d2-187428f14c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:30.701495 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bff58-2f6b-4af5-88d2-187428f14c07 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:30.701781 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bff58-2f6b-4af5-88d2-187428f14c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:30.702050 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bff58-2f6b-4af5-88d2-187428f14c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:30.703881 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-b61bff58-2f6b-4af5-88d2-187428f14c07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:30.778250 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bff58-2f6b-4af5-88d2-187428f14c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:30.779787 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bff58-2f6b-4af5-88d2-187428f14c07 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:30.791244 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1433/2880] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:36.626099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb3132-86c9-4b66-b2b2-2eb4138338d8 service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:36.627501 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cdeb3132-86c9-4b66-b2b2-2eb4138338d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:36.699256 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb3132-86c9-4b66-b2b2-2eb4138338d8 service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:36.704439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc6758-ee8d-4118-b633-64ff1b3c3b7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:36.705041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc6758-ee8d-4118-b633-64ff1b3c3b7d service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:36.705315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc6758-ee8d-4118-b633-64ff1b3c3b7d service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:36.705614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc6758-ee8d-4118-b633-64ff1b3c3b7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:36.707456 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b3bc6758-ee8d-4118-b633-64ff1b3c3b7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:36.719577 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc6758-ee8d-4118-b633-64ff1b3c3b7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:36.721140 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc6758-ee8d-4118-b633-64ff1b3c3b7d service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:36.731649 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1448/2881] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:38.269336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec5388-0400-4438-8e8d-47c78448ad54 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:38.269818 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec5388-0400-4438-8e8d-47c78448ad54 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:38.269918 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec5388-0400-4438-8e8d-47c78448ad54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:38.320570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-adec5388-0400-4438-8e8d-47c78448ad54 None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:38.321826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-adec5388-0400-4438-8e8d-47c78448ad54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:38.387011 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1434/2882] 10.222.0.62 () {58 vars in 1009 bytes} [Fri Sep 9 12:19:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:19:38.432426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc6758-ee8d-4118-b633-64ff1b3c3b7d service placement] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:38.433817 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b3bc6758-ee8d-4118-b633-64ff1b3c3b7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:38.524072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc6758-ee8d-4118-b633-64ff1b3c3b7d service placement] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:38.529199 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0237dad-230e-41d2-abf6-0e5aa4d2778d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:38.529805 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0237dad-230e-41d2-abf6-0e5aa4d2778d service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:38.530086 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0237dad-230e-41d2-abf6-0e5aa4d2778d service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:38.530400 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0237dad-230e-41d2-abf6-0e5aa4d2778d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:38.538413 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0237dad-230e-41d2-abf6-0e5aa4d2778d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:38.539977 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0237dad-230e-41d2-abf6-0e5aa4d2778d service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:38.548041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1449/2883] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:19:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:19:39.059526 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279ae67-694b-423a-a0af-7b687aad31a7 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.060037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279ae67-694b-423a-a0af-7b687aad31a7 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.060543 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279ae67-694b-423a-a0af-7b687aad31a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.083970 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-7279ae67-694b-423a-a0af-7b687aad31a7 None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:39.085220 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-7279ae67-694b-423a-a0af-7b687aad31a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:39.094428 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1435/2884] 10.222.0.62 () {58 vars in 967 bytes} [Fri Sep 9 12:19:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:39.109820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0237dad-230e-41d2-abf6-0e5aa4d2778d service swift] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:39.121087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0237dad-230e-41d2-abf6-0e5aa4d2778d service swift] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:39.126433 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d9ba8-6a51-4478-bfc4-1699fb4926c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:39.127073 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d9ba8-6a51-4478-bfc4-1699fb4926c7 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.127334 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d9ba8-6a51-4478-bfc4-1699fb4926c7 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.127638 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d9ba8-6a51-4478-bfc4-1699fb4926c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.135647 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d9ba8-6a51-4478-bfc4-1699fb4926c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:39.137208 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d9ba8-6a51-4478-bfc4-1699fb4926c7 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:39.145373 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1450/2885] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:19:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:19:39.372829 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964d002-8172-4b33-816a-f9685ca5f50e None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.373360 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964d002-8172-4b33-816a-f9685ca5f50e None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.373360 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964d002-8172-4b33-816a-f9685ca5f50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.398488 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-6964d002-8172-4b33-816a-f9685ca5f50e None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:39.399568 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-6964d002-8172-4b33-816a-f9685ca5f50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:39.406883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1436/2886] 10.222.0.62 () {58 vars in 1009 bytes} [Fri Sep 9 12:19:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:39.419688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d9ba8-6a51-4478-bfc4-1699fb4926c7 service swift] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:39.430247 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d9ba8-6a51-4478-bfc4-1699fb4926c7 service swift] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:39.436991 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9b595-87a6-4d2c-86ce-129de44fe745 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:39.437566 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9b595-87a6-4d2c-86ce-129de44fe745 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.437851 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9b595-87a6-4d2c-86ce-129de44fe745 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.438144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9b595-87a6-4d2c-86ce-129de44fe745 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.446264 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9b595-87a6-4d2c-86ce-129de44fe745 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:39.447827 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9b595-87a6-4d2c-86ce-129de44fe745 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:39.455851 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1451/2887] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:19:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:19:39.776578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc243dec-7900-4879-99e3-3cff12e7e4e7 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.776897 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc243dec-7900-4879-99e3-3cff12e7e4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.777145 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc243dec-7900-4879-99e3-3cff12e7e4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.796492 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-dc243dec-7900-4879-99e3-3cff12e7e4e7 None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:39.797681 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-dc243dec-7900-4879-99e3-3cff12e7e4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:39.804957 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1437/2888] 10.222.0.62 () {58 vars in 967 bytes} [Fri Sep 9 12:19:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:39.817120 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9b595-87a6-4d2c-86ce-129de44fe745 service swift] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:39.828253 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9b595-87a6-4d2c-86ce-129de44fe745 service swift] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:39.833263 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003019e-a411-4d48-a861-686544e2f8ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:39.833862 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003019e-a411-4d48-a861-686544e2f8ea service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.834169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003019e-a411-4d48-a861-686544e2f8ea service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.834446 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003019e-a411-4d48-a861-686544e2f8ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.843100 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003019e-a411-4d48-a861-686544e2f8ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:39.844677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003019e-a411-4d48-a861-686544e2f8ea service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:39.853061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1452/2889] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:19:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:19:44.129666 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc243dec-7900-4879-99e3-3cff12e7e4e7 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:44.139524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=100 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:44.225111 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc243dec-7900-4879-99e3-3cff12e7e4e7 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:44.232908 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272794cb-a439-4d1a-b32c-2c501d91204a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:44.233764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272794cb-a439-4d1a-b32c-2c501d91204a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:44.234277 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272794cb-a439-4d1a-b32c-2c501d91204a admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:44.234699 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272794cb-a439-4d1a-b32c-2c501d91204a admin admin] PATH_INFO: `/v3/users/392ffc42301e4752beb3046b1ffca33f` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:44.236338 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272794cb-a439-4d1a-b32c-2c501d91204a admin admin] RBAC: Authorizing `identity:delete_user(user_id=392ffc42301e4752beb3046b1ffca33f)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:44.238529 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272794cb-a439-4d1a-b32c-2c501d91204a admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:44.318343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-272794cb-a439-4d1a-b32c-2c501d91204a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '392ffc42301e4752beb3046b1ffca33f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:44.338137 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-272794cb-a439-4d1a-b32c-2c501d91204a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '392ffc42301e4752beb3046b1ffca33f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:44.352941 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-272794cb-a439-4d1a-b32c-2c501d91204a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '392ffc42301e4752beb3046b1ffca33f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:44.360520 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-272794cb-a439-4d1a-b32c-2c501d91204a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '392ffc42301e4752beb3046b1ffca33f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:44.361275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-272794cb-a439-4d1a-b32c-2c501d91204a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '392ffc42301e4752beb3046b1ffca33f'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:44.367644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1438/2890] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:44 2022] DELETE /identity/v3/users/392ffc42301e4752beb3046b1ffca33f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:44.372144 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd536f7-921f-4afd-8df9-982760074b3a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:44.372914 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd536f7-921f-4afd-8df9-982760074b3a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:44.373128 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd536f7-921f-4afd-8df9-982760074b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:44.395652 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-acd536f7-921f-4afd-8df9-982760074b3a None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:44.396879 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-acd536f7-921f-4afd-8df9-982760074b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:44.412534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:44.467386 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1453/2891] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:44.478697 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272794cb-a439-4d1a-b32c-2c501d91204a admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:44.480115 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-272794cb-a439-4d1a-b32c-2c501d91204a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:44.549457 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272794cb-a439-4d1a-b32c-2c501d91204a admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:44.554883 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5ddc0-75aa-4f5b-a4e5-7793cfdcbf2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:44.555571 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5ddc0-75aa-4f5b-a4e5-7793cfdcbf2d service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:44.555848 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5ddc0-75aa-4f5b-a4e5-7793cfdcbf2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:44.556114 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5ddc0-75aa-4f5b-a4e5-7793cfdcbf2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:44.566949 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5ddc0-75aa-4f5b-a4e5-7793cfdcbf2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:44.568038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5ddc0-75aa-4f5b-a4e5-7793cfdcbf2d service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:44.579517 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1439/2892] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:44.781887 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd536f7-921f-4afd-8df9-982760074b3a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:44.784097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-acd536f7-921f-4afd-8df9-982760074b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:44.804396 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd536f7-921f-4afd-8df9-982760074b3a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:44.814062 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cceff-720c-4682-844f-1dda62707d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:44.814915 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cceff-720c-4682-844f-1dda62707d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:44.815667 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cceff-720c-4682-844f-1dda62707d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:44.816087 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cceff-720c-4682-844f-1dda62707d59 admin admin] PATH_INFO: `/v3/projects/781d68e5617d4fcd838836192767453e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:44.817274 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1cceff-720c-4682-844f-1dda62707d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=781d68e5617d4fcd838836192767453e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:44.819626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1cceff-720c-4682-844f-1dda62707d59 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:44.870886 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3c1cceff-720c-4682-844f-1dda62707d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '781d68e5617d4fcd838836192767453e', 'request_id': 'req-3c1cceff-720c-4682-844f-1dda62707d59'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:44.880642 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3c1cceff-720c-4682-844f-1dda62707d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '781d68e5617d4fcd838836192767453e', 'request_id': 'req-3c1cceff-720c-4682-844f-1dda62707d59'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:44.891347 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3c1cceff-720c-4682-844f-1dda62707d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '781d68e5617d4fcd838836192767453e', 'request_id': 'req-3c1cceff-720c-4682-844f-1dda62707d59'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:44.903006 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3c1cceff-720c-4682-844f-1dda62707d59 admin admin] The token cache is being invalidate because project 781d68e5617d4fcd838836192767453e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:44.903293 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3c1cceff-720c-4682-844f-1dda62707d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:44.903949 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3c1cceff-720c-4682-844f-1dda62707d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:44.905771 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1454/2893] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:44 2022] DELETE /identity/v3/projects/781d68e5617d4fcd838836192767453e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:54.574100 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:54.900449 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:55.743452 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d82cb-f3f6-4dd9-a7e7-b3fc6fd57746 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.744094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d82cb-f3f6-4dd9-a7e7-b3fc6fd57746 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.744455 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d82cb-f3f6-4dd9-a7e7-b3fc6fd57746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.764522 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-f06d82cb-f3f6-4dd9-a7e7-b3fc6fd57746 None None] MFA Rules not processed for user `01834ce862ae47cc85668b0a1c66f587`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:55.765716 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f06d82cb-f3f6-4dd9-a7e7-b3fc6fd57746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:55.821007 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1440/2894] 10.222.0.62 () {58 vars in 967 bytes} [Fri Sep 9 12:19:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:55.833834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cceff-720c-4682-844f-1dda62707d59 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:55.835599 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3c1cceff-720c-4682-844f-1dda62707d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:55.910574 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cceff-720c-4682-844f-1dda62707d59 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:55.915490 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b8e1b-ced6-4286-9b35-5e9638add808 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2de52919e54bf9bcc352909c9f905f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:55.916099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b8e1b-ced6-4286-9b35-5e9638add808 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.916375 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b8e1b-ced6-4286-9b35-5e9638add808 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.916646 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b8e1b-ced6-4286-9b35-5e9638add808 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.924716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b8e1b-ced6-4286-9b35-5e9638add808 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:55.926328 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b8e1b-ced6-4286-9b35-5e9638add808 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:55.934911 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1455/2895] 10.222.0.62 () {60 vars in 1384 bytes} [Fri Sep 9 12:19:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:20:00.022333 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d82cb-f3f6-4dd9-a7e7-b3fc6fd57746 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:00.023778 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f06d82cb-f3f6-4dd9-a7e7-b3fc6fd57746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:00.137491 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d82cb-f3f6-4dd9-a7e7-b3fc6fd57746 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:00.144313 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04498854-86b8-494b-b2b6-7601e07cbeab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:00.145036 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04498854-86b8-494b-b2b6-7601e07cbeab service placement] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:00.145524 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04498854-86b8-494b-b2b6-7601e07cbeab service placement] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:00.145890 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04498854-86b8-494b-b2b6-7601e07cbeab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:00.148375 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-04498854-86b8-494b-b2b6-7601e07cbeab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:00.167803 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04498854-86b8-494b-b2b6-7601e07cbeab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:00.170156 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04498854-86b8-494b-b2b6-7601e07cbeab service placement] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:00.182636 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1441/2896] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:20:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:20:02.488578 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b8e1b-ced6-4286-9b35-5e9638add808 service swift] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:02.490048 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bb1b8e1b-ced6-4286-9b35-5e9638add808 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:02.548355 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b8e1b-ced6-4286-9b35-5e9638add808 service swift] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:02.552036 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda2249-d11d-45a3-828f-3a2fa414a794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:02.552634 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda2249-d11d-45a3-828f-3a2fa414a794 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:02.552906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda2249-d11d-45a3-828f-3a2fa414a794 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:02.553174 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda2249-d11d-45a3-828f-3a2fa414a794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:02.554898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0bda2249-d11d-45a3-828f-3a2fa414a794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:02.618735 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bda2249-d11d-45a3-828f-3a2fa414a794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:02.620159 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bda2249-d11d-45a3-828f-3a2fa414a794 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:02.630089 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1456/2897] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:20:03.460972 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04498854-86b8-494b-b2b6-7601e07cbeab service placement] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:03.465422 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-04498854-86b8-494b-b2b6-7601e07cbeab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:03.484429 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04498854-86b8-494b-b2b6-7601e07cbeab service placement] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:03.489029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517c0d3-34a3-4bb2-a777-e7b1d4bb8e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:03.489644 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517c0d3-34a3-4bb2-a777-e7b1d4bb8e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:03.489917 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517c0d3-34a3-4bb2-a777-e7b1d4bb8e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:03.490216 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517c0d3-34a3-4bb2-a777-e7b1d4bb8e0b admin admin] PATH_INFO: `/v3/users/cb72d25947724d028d948292925258d9` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:03.491378 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517c0d3-34a3-4bb2-a777-e7b1d4bb8e0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb72d25947724d028d948292925258d9)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:03.493255 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517c0d3-34a3-4bb2-a777-e7b1d4bb8e0b admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:03.542284 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d517c0d3-34a3-4bb2-a777-e7b1d4bb8e0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb72d25947724d028d948292925258d9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:03.549653 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d517c0d3-34a3-4bb2-a777-e7b1d4bb8e0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb72d25947724d028d948292925258d9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:03.561744 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d517c0d3-34a3-4bb2-a777-e7b1d4bb8e0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb72d25947724d028d948292925258d9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:03.568835 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d517c0d3-34a3-4bb2-a777-e7b1d4bb8e0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb72d25947724d028d948292925258d9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:03.569586 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-d517c0d3-34a3-4bb2-a777-e7b1d4bb8e0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb72d25947724d028d948292925258d9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:03.575575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1442/2898] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:20:03 2022] DELETE /identity/v3/users/cb72d25947724d028d948292925258d9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:03.757358 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda2249-d11d-45a3-828f-3a2fa414a794 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:03.767941 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda2249-d11d-45a3-828f-3a2fa414a794 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:03.809360 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:03.828119 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b16760-b74c-4523-9d33-4734c231d47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:03.828764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b16760-b74c-4523-9d33-4734c231d47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:03.829041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b16760-b74c-4523-9d33-4734c231d47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:03.829318 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b16760-b74c-4523-9d33-4734c231d47f admin admin] PATH_INFO: `/v3/projects/68a3919f2172442fa280c380a7bff271` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:03.830462 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b16760-b74c-4523-9d33-4734c231d47f admin admin] RBAC: Authorizing `identity:delete_project(project_id=68a3919f2172442fa280c380a7bff271)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:03.832356 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b16760-b74c-4523-9d33-4734c231d47f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:03.879517 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-39b16760-b74c-4523-9d33-4734c231d47f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68a3919f2172442fa280c380a7bff271', 'request_id': 'req-39b16760-b74c-4523-9d33-4734c231d47f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:03.888430 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-39b16760-b74c-4523-9d33-4734c231d47f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68a3919f2172442fa280c380a7bff271', 'request_id': 'req-39b16760-b74c-4523-9d33-4734c231d47f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:03.894592 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-39b16760-b74c-4523-9d33-4734c231d47f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68a3919f2172442fa280c380a7bff271', 'request_id': 'req-39b16760-b74c-4523-9d33-4734c231d47f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:03.905139 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-39b16760-b74c-4523-9d33-4734c231d47f admin admin] The token cache is being invalidate because project 68a3919f2172442fa280c380a7bff271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:20:03.905427 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-39b16760-b74c-4523-9d33-4734c231d47f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:03.906061 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-39b16760-b74c-4523-9d33-4734c231d47f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:03.907595 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1457/2899] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:20:03 2022] DELETE /identity/v3/projects/68a3919f2172442fa280c380a7bff271 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:03.911627 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517c0d3-34a3-4bb2-a777-e7b1d4bb8e0b admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:03.913031 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-d517c0d3-34a3-4bb2-a777-e7b1d4bb8e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:03.926728 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517c0d3-34a3-4bb2-a777-e7b1d4bb8e0b admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:03.930219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:03.984414 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35d44b-5339-444a-8049-d4fda114c7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:03.985043 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35d44b-5339-444a-8049-d4fda114c7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:03.985323 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35d44b-5339-444a-8049-d4fda114c7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:03.985595 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35d44b-5339-444a-8049-d4fda114c7c1 admin admin] PATH_INFO: `/v3/users/f819395e7004410f9c0cd83bd7f25e4d` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:03.986754 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35d44b-5339-444a-8049-d4fda114c7c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f819395e7004410f9c0cd83bd7f25e4d)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:03.988680 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35d44b-5339-444a-8049-d4fda114c7c1 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:04.035478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2a35d44b-5339-444a-8049-d4fda114c7c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f819395e7004410f9c0cd83bd7f25e4d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.042346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2a35d44b-5339-444a-8049-d4fda114c7c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f819395e7004410f9c0cd83bd7f25e4d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.054500 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2a35d44b-5339-444a-8049-d4fda114c7c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f819395e7004410f9c0cd83bd7f25e4d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.061891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2a35d44b-5339-444a-8049-d4fda114c7c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f819395e7004410f9c0cd83bd7f25e4d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.062573 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-2a35d44b-5339-444a-8049-d4fda114c7c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f819395e7004410f9c0cd83bd7f25e4d'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.068503 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1443/2900] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:20:03 2022] DELETE /identity/v3/users/f819395e7004410f9c0cd83bd7f25e4d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:04.244034 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b16760-b74c-4523-9d33-4734c231d47f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:04.245658 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-39b16760-b74c-4523-9d33-4734c231d47f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:04.259731 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b16760-b74c-4523-9d33-4734c231d47f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:04.318479 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13308437-425d-412e-9440-f941cb4dfb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:04.319037 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13308437-425d-412e-9440-f941cb4dfb70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:04.319311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13308437-425d-412e-9440-f941cb4dfb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:04.319579 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13308437-425d-412e-9440-f941cb4dfb70 admin admin] PATH_INFO: `/v3/projects/4e7db0beb2fa4e4a96416bb3673763f3` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:04.320686 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13308437-425d-412e-9440-f941cb4dfb70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e7db0beb2fa4e4a96416bb3673763f3)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:04.323166 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13308437-425d-412e-9440-f941cb4dfb70 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:04.370010 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-13308437-425d-412e-9440-f941cb4dfb70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e7db0beb2fa4e4a96416bb3673763f3', 'request_id': 'req-13308437-425d-412e-9440-f941cb4dfb70'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.377616 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-13308437-425d-412e-9440-f941cb4dfb70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e7db0beb2fa4e4a96416bb3673763f3', 'request_id': 'req-13308437-425d-412e-9440-f941cb4dfb70'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.383299 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-13308437-425d-412e-9440-f941cb4dfb70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e7db0beb2fa4e4a96416bb3673763f3', 'request_id': 'req-13308437-425d-412e-9440-f941cb4dfb70'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.391790 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:04.395916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-13308437-425d-412e-9440-f941cb4dfb70 admin admin] The token cache is being invalidate because project 4e7db0beb2fa4e4a96416bb3673763f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:20:04.396209 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-13308437-425d-412e-9440-f941cb4dfb70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.397218 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-13308437-425d-412e-9440-f941cb4dfb70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.399178 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1458/2901] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:20:04 2022] DELETE /identity/v3/projects/4e7db0beb2fa4e4a96416bb3673763f3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:14.065076 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=50,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:23.703705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35d44b-5339-444a-8049-d4fda114c7c1 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:23.705247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-2a35d44b-5339-444a-8049-d4fda114c7c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:23.718853 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35d44b-5339-444a-8049-d4fda114c7c1 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:23.778821 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010d434-6035-4e99-8782-04e6b1eb24bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:23.779366 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010d434-6035-4e99-8782-04e6b1eb24bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:23.779642 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010d434-6035-4e99-8782-04e6b1eb24bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:23.779909 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010d434-6035-4e99-8782-04e6b1eb24bc admin admin] PATH_INFO: `/v3/users/f849563115ec4a4e859a7395a03a561a` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:23.781060 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010d434-6035-4e99-8782-04e6b1eb24bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f849563115ec4a4e859a7395a03a561a)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:23.783018 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010d434-6035-4e99-8782-04e6b1eb24bc admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:23.830104 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4010d434-6035-4e99-8782-04e6b1eb24bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f849563115ec4a4e859a7395a03a561a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:23.837912 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4010d434-6035-4e99-8782-04e6b1eb24bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f849563115ec4a4e859a7395a03a561a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:23.849823 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4010d434-6035-4e99-8782-04e6b1eb24bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f849563115ec4a4e859a7395a03a561a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:23.856773 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4010d434-6035-4e99-8782-04e6b1eb24bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f849563115ec4a4e859a7395a03a561a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:23.857521 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-4010d434-6035-4e99-8782-04e6b1eb24bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f849563115ec4a4e859a7395a03a561a'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:23.863443 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1444/2902] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:20:23 2022] DELETE /identity/v3/users/f849563115ec4a4e859a7395a03a561a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:24.036489 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13308437-425d-412e-9440-f941cb4dfb70 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:24.037925 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-13308437-425d-412e-9440-f941cb4dfb70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:24.052632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13308437-425d-412e-9440-f941cb4dfb70 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:24.109784 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:24.110453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:24.110729 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:24.110996 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca admin admin] PATH_INFO: `/v3/projects/770c076806c64cd78d35f095ee489bcf` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:24.112091 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=770c076806c64cd78d35f095ee489bcf)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:24.113963 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:24.160164 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '770c076806c64cd78d35f095ee489bcf', 'request_id': 'req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:24.167577 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '770c076806c64cd78d35f095ee489bcf', 'request_id': 'req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:24.173256 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '770c076806c64cd78d35f095ee489bcf', 'request_id': 'req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:24.184218 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca admin admin] The token cache is being invalidate because project 770c076806c64cd78d35f095ee489bcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:20:24.184507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:24.185154 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:24.186680 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1459/2903] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:20:24 2022] DELETE /identity/v3/projects/770c076806c64cd78d35f095ee489bcf => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:33.859952 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:34.181288 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:22:02.208811 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010d434-6035-4e99-8782-04e6b1eb24bc admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:22:02.210380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-4010d434-6035-4e99-8782-04e6b1eb24bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:22:02.268179 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010d434-6035-4e99-8782-04e6b1eb24bc admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:22:02.316615 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f581b9-29da-4b17-9ace-98457c3a165f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:22:02.317219 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f581b9-29da-4b17-9ace-98457c3a165f service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:22:02.317499 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f581b9-29da-4b17-9ace-98457c3a165f service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:22:02.317771 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f581b9-29da-4b17-9ace-98457c3a165f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:22:02.319583 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-e8f581b9-29da-4b17-9ace-98457c3a165f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:22:02.389622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f581b9-29da-4b17-9ace-98457c3a165f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:22:02.391106 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f581b9-29da-4b17-9ace-98457c3a165f service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:22:02.402610 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1445/2904] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:22:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:22:03.595876 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:22:03.597450 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:22:03.610352 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2338b1-bed9-4ae9-a1cb-e539be0a17ca admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:22:03.614054 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd214336-5408-4eb3-98fd-37a3c1593621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:22:03.614675 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd214336-5408-4eb3-98fd-37a3c1593621 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:22:03.614947 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd214336-5408-4eb3-98fd-37a3c1593621 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:22:03.615212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd214336-5408-4eb3-98fd-37a3c1593621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:22:03.616889 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cd214336-5408-4eb3-98fd-37a3c1593621 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:22:03.631543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd214336-5408-4eb3-98fd-37a3c1593621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:22:03.632951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd214336-5408-4eb3-98fd-37a3c1593621 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:22:03.644029 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1460/2905] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:22:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:22:12.397299 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=74 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:22:12.441185 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f581b9-29da-4b17-9ace-98457c3a165f service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:22:12.449963 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f581b9-29da-4b17-9ace-98457c3a165f service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:22:12.454667 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c2f2a-7f24-4807-8ac8-b7c4fc924e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:22:12.455296 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c2f2a-7f24-4807-8ac8-b7c4fc924e37 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:22:12.455570 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c2f2a-7f24-4807-8ac8-b7c4fc924e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:22:12.455841 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c2f2a-7f24-4807-8ac8-b7c4fc924e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:22:12.457613 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-f47c2f2a-7f24-4807-8ac8-b7c4fc924e37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:22:12.468170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c2f2a-7f24-4807-8ac8-b7c4fc924e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:22:12.469530 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c2f2a-7f24-4807-8ac8-b7c4fc924e37 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:22:12.478029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1446/2906] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:22:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:22:13.639353 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:22:15.940633 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd214336-5408-4eb3-98fd-37a3c1593621 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:22:15.949498 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd214336-5408-4eb3-98fd-37a3c1593621 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:22:15.954365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fdd12-495c-4e74-aac9-1b4f5f89a2f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:22:15.955267 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fdd12-495c-4e74-aac9-1b4f5f89a2f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:22:15.955815 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fdd12-495c-4e74-aac9-1b4f5f89a2f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:22:15.956173 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fdd12-495c-4e74-aac9-1b4f5f89a2f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:22:15.958554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-721fdd12-495c-4e74-aac9-1b4f5f89a2f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:22:15.969725 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fdd12-495c-4e74-aac9-1b4f5f89a2f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:22:15.971185 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fdd12-495c-4e74-aac9-1b4f5f89a2f0 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:22:15.979584 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1461/2907] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:22:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:22:16.084621 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c2f2a-7f24-4807-8ac8-b7c4fc924e37 service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:22:16.094131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c2f2a-7f24-4807-8ac8-b7c4fc924e37 service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:22:16.097840 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc45849-ebea-4d7c-af97-8bb2065e44e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:22:16.098546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc45849-ebea-4d7c-af97-8bb2065e44e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:22:16.098893 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc45849-ebea-4d7c-af97-8bb2065e44e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:22:16.099263 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc45849-ebea-4d7c-af97-8bb2065e44e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:22:16.101084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0fc45849-ebea-4d7c-af97-8bb2065e44e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:22:16.115546 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc45849-ebea-4d7c-af97-8bb2065e44e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:22:16.116975 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc45849-ebea-4d7c-af97-8bb2065e44e2 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:22:16.128380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1447/2908] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:22:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:22:25.975873 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:22:26.123130 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=12 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:23:57.284609 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fdd12-495c-4e74-aac9-1b4f5f89a2f0 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:23:57.286183 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-721fdd12-495c-4e74-aac9-1b4f5f89a2f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:57.297136 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fdd12-495c-4e74-aac9-1b4f5f89a2f0 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:23:57.300830 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc276e4-1264-46c5-bac3-e0faac26e885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:23:57.301426 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc276e4-1264-46c5-bac3-e0faac26e885 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:23:57.301704 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc276e4-1264-46c5-bac3-e0faac26e885 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:23:57.301975 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc276e4-1264-46c5-bac3-e0faac26e885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:23:57.303748 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ffc276e4-1264-46c5-bac3-e0faac26e885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:57.362080 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc276e4-1264-46c5-bac3-e0faac26e885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:23:57.363514 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc276e4-1264-46c5-bac3-e0faac26e885 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:23:57.372717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1462/2909] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:23:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:23:58.214662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc45849-ebea-4d7c-af97-8bb2065e44e2 service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:23:58.216459 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0fc45849-ebea-4d7c-af97-8bb2065e44e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:58.242424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc45849-ebea-4d7c-af97-8bb2065e44e2 service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:23:58.248403 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d0818-f003-404c-b262-1df39d9fa7f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:23:58.249016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d0818-f003-404c-b262-1df39d9fa7f6 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:23:58.249297 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d0818-f003-404c-b262-1df39d9fa7f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:23:58.249569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d0818-f003-404c-b262-1df39d9fa7f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:23:58.252063 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-597d0818-f003-404c-b262-1df39d9fa7f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:58.265340 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d0818-f003-404c-b262-1df39d9fa7f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:23:58.267633 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d0818-f003-404c-b262-1df39d9fa7f6 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:23:58.279891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1448/2910] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:23:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:02.101936 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc276e4-1264-46c5-bac3-e0faac26e885 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:02.110799 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc276e4-1264-46c5-bac3-e0faac26e885 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:02.114480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b0155-263d-4a02-b980-7b89cd0fd928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:02.115074 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b0155-263d-4a02-b980-7b89cd0fd928 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:02.115351 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b0155-263d-4a02-b980-7b89cd0fd928 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:02.115623 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b0155-263d-4a02-b980-7b89cd0fd928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:02.117411 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ce9b0155-263d-4a02-b980-7b89cd0fd928 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:02.181000 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b0155-263d-4a02-b980-7b89cd0fd928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:02.182455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b0155-263d-4a02-b980-7b89cd0fd928 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:02.192709 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1463/2911] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:08.274695 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=6 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:08.308166 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d0818-f003-404c-b262-1df39d9fa7f6 service nova] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:08.309661 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-597d0818-f003-404c-b262-1df39d9fa7f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:08.324073 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d0818-f003-404c-b262-1df39d9fa7f6 service nova] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:08.328723 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54992521-128c-4eff-9efe-13d58762f071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:08.329390 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54992521-128c-4eff-9efe-13d58762f071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:08.329619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54992521-128c-4eff-9efe-13d58762f071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:08.329888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54992521-128c-4eff-9efe-13d58762f071 admin admin] PATH_INFO: `/v3/users/7e4d42a3879f4ec2b422f4bc8dd2dea9` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:08.331062 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54992521-128c-4eff-9efe-13d58762f071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e4d42a3879f4ec2b422f4bc8dd2dea9)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:08.332965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54992521-128c-4eff-9efe-13d58762f071 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:08.383736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-54992521-128c-4eff-9efe-13d58762f071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e4d42a3879f4ec2b422f4bc8dd2dea9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:08.391187 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-54992521-128c-4eff-9efe-13d58762f071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e4d42a3879f4ec2b422f4bc8dd2dea9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:08.405966 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-54992521-128c-4eff-9efe-13d58762f071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e4d42a3879f4ec2b422f4bc8dd2dea9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:08.412820 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-54992521-128c-4eff-9efe-13d58762f071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e4d42a3879f4ec2b422f4bc8dd2dea9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:08.413483 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-54992521-128c-4eff-9efe-13d58762f071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e4d42a3879f4ec2b422f4bc8dd2dea9'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:08.419489 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1449/2912] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:24:08 2022] DELETE /identity/v3/users/7e4d42a3879f4ec2b422f4bc8dd2dea9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:08.591653 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b0155-263d-4a02-b980-7b89cd0fd928 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:08.602099 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b0155-263d-4a02-b980-7b89cd0fd928 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:08.660780 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:08.661406 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:08.661680 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:08.661950 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0 admin admin] PATH_INFO: `/v3/projects/26d88da533a4432a97eb9644ce21fcfe` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:08.663085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26d88da533a4432a97eb9644ce21fcfe)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:08.664972 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:08.701551 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:08.712523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26d88da533a4432a97eb9644ce21fcfe', 'request_id': 'req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:08.720409 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26d88da533a4432a97eb9644ce21fcfe', 'request_id': 'req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:08.725981 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26d88da533a4432a97eb9644ce21fcfe', 'request_id': 'req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:08.737480 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0 admin admin] The token cache is being invalidate because project 26d88da533a4432a97eb9644ce21fcfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:24:08.737769 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:08.738434 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:08.739965 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1464/2913] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:24:08 2022] DELETE /identity/v3/projects/26d88da533a4432a97eb9644ce21fcfe => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:08.744143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54992521-128c-4eff-9efe-13d58762f071 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:08.745661 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-54992521-128c-4eff-9efe-13d58762f071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:08.759221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54992521-128c-4eff-9efe-13d58762f071 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:08.816038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb35540-4955-4851-a9d9-3d4f202c09bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:08.816662 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb35540-4955-4851-a9d9-3d4f202c09bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:08.816938 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb35540-4955-4851-a9d9-3d4f202c09bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:08.817205 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb35540-4955-4851-a9d9-3d4f202c09bb admin admin] PATH_INFO: `/v3/users/6648cfb8e2fd4e0894f09223104df8e2` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:08.818358 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb35540-4955-4851-a9d9-3d4f202c09bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6648cfb8e2fd4e0894f09223104df8e2)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:08.820248 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb35540-4955-4851-a9d9-3d4f202c09bb admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:08.868618 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7cb35540-4955-4851-a9d9-3d4f202c09bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6648cfb8e2fd4e0894f09223104df8e2'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:08.876240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7cb35540-4955-4851-a9d9-3d4f202c09bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6648cfb8e2fd4e0894f09223104df8e2'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:08.888547 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7cb35540-4955-4851-a9d9-3d4f202c09bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6648cfb8e2fd4e0894f09223104df8e2'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:08.890552 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:08.896281 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7cb35540-4955-4851-a9d9-3d4f202c09bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6648cfb8e2fd4e0894f09223104df8e2'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:08.897622 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-7cb35540-4955-4851-a9d9-3d4f202c09bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6648cfb8e2fd4e0894f09223104df8e2'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:08.904331 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1450/2914] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:24:08 2022] DELETE /identity/v3/users/6648cfb8e2fd4e0894f09223104df8e2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:09.080548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:09.082258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:09.109985 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a221d-649c-41e3-98bc-fe65d5cf6bd0 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:09.170455 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a51ee-3292-4f50-87f5-f6ec87772f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:09.171205 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a51ee-3292-4f50-87f5-f6ec87772f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:09.171479 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a51ee-3292-4f50-87f5-f6ec87772f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:09.171747 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a51ee-3292-4f50-87f5-f6ec87772f88 admin admin] PATH_INFO: `/v3/projects/7b33b931648b49c198607d02eca7c406` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:09.173534 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a51ee-3292-4f50-87f5-f6ec87772f88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b33b931648b49c198607d02eca7c406)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:09.175507 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a51ee-3292-4f50-87f5-f6ec87772f88 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:09.228037 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f51a51ee-3292-4f50-87f5-f6ec87772f88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b33b931648b49c198607d02eca7c406', 'request_id': 'req-f51a51ee-3292-4f50-87f5-f6ec87772f88'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:09.235439 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f51a51ee-3292-4f50-87f5-f6ec87772f88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b33b931648b49c198607d02eca7c406', 'request_id': 'req-f51a51ee-3292-4f50-87f5-f6ec87772f88'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:09.241014 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f51a51ee-3292-4f50-87f5-f6ec87772f88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b33b931648b49c198607d02eca7c406', 'request_id': 'req-f51a51ee-3292-4f50-87f5-f6ec87772f88'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:09.252626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f51a51ee-3292-4f50-87f5-f6ec87772f88 admin admin] The token cache is being invalidate because project 7b33b931648b49c198607d02eca7c406 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:24:09.252915 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f51a51ee-3292-4f50-87f5-f6ec87772f88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:09.253543 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f51a51ee-3292-4f50-87f5-f6ec87772f88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:09.255060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1465/2915] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:24:09 2022] DELETE /identity/v3/projects/7b33b931648b49c198607d02eca7c406 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:09.259947 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30734774-d67a-4e76-a1ee-d86eeb77dfb4 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:09.260231 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30734774-d67a-4e76-a1ee-d86eeb77dfb4 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:09.260507 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30734774-d67a-4e76-a1ee-d86eeb77dfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:09.280218 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-30734774-d67a-4e76-a1ee-d86eeb77dfb4 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:09.282013 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-30734774-d67a-4e76-a1ee-d86eeb77dfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:09.340172 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1451/2916] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:09.348612 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aac1ca-f643-4ad1-9e3b-7bc7fd2e41b7 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:09.348897 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aac1ca-f643-4ad1-9e3b-7bc7fd2e41b7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:09.349169 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aac1ca-f643-4ad1-9e3b-7bc7fd2e41b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:09.368573 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-73aac1ca-f643-4ad1-9e3b-7bc7fd2e41b7 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:09.369769 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-73aac1ca-f643-4ad1-9e3b-7bc7fd2e41b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:09.377814 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1466/2917] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:24:09.381940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30734774-d67a-4e76-a1ee-d86eeb77dfb4 None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:09.392085 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30734774-d67a-4e76-a1ee-d86eeb77dfb4 None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:09.396692 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd68ab-cd78-4b4d-9162-456642506a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:09.397308 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd68ab-cd78-4b4d-9162-456642506a6e admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:09.397575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd68ab-cd78-4b4d-9162-456642506a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:09.397854 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd68ab-cd78-4b4d-9162-456642506a6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:09.398545 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd68ab-cd78-4b4d-9162-456642506a6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81447) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:24:09.398851 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd68ab-cd78-4b4d-9162-456642506a6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:09.400444 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd68ab-cd78-4b4d-9162-456642506a6e admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:09.412675 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1452/2918] 10.222.0.62 () {58 vars in 1153 bytes} [Fri Sep 9 12:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:24:09.416820 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aac1ca-f643-4ad1-9e3b-7bc7fd2e41b7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:09.426152 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aac1ca-f643-4ad1-9e3b-7bc7fd2e41b7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:09.430951 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30590869-a108-41ab-9aa6-8d5628fe4e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:09.431547 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30590869-a108-41ab-9aa6-8d5628fe4e7b admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:09.431886 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30590869-a108-41ab-9aa6-8d5628fe4e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:09.432094 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30590869-a108-41ab-9aa6-8d5628fe4e7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:09.432733 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30590869-a108-41ab-9aa6-8d5628fe4e7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:09.434651 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30590869-a108-41ab-9aa6-8d5628fe4e7b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:09.477162 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1467/2919] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:24:09 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:09.479346 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd68ab-cd78-4b4d-9162-456642506a6e admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:09.489373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd68ab-cd78-4b4d-9162-456642506a6e admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:09.546568 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba7497-8b16-446d-85c8-18dd60a1a828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:09.547178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba7497-8b16-446d-85c8-18dd60a1a828 admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:09.547460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba7497-8b16-446d-85c8-18dd60a1a828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:09.547760 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba7497-8b16-446d-85c8-18dd60a1a828 admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:09.548431 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba7497-8b16-446d-85c8-18dd60a1a828 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:09.550460 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba7497-8b16-446d-85c8-18dd60a1a828 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:09.573578 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1453/2920] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:24:09 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:09.576041 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30590869-a108-41ab-9aa6-8d5628fe4e7b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:09.587002 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30590869-a108-41ab-9aa6-8d5628fe4e7b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:09.591618 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf38c5-a022-4ca8-97ea-e85890a419f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:09.592213 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf38c5-a022-4ca8-97ea-e85890a419f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:09.592773 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf38c5-a022-4ca8-97ea-e85890a419f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:09.592773 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf38c5-a022-4ca8-97ea-e85890a419f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:09.593377 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf38c5-a022-4ca8-97ea-e85890a419f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:09.595473 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf38c5-a022-4ca8-97ea-e85890a419f3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:09.605437 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1468/2921] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:09.609351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba7497-8b16-446d-85c8-18dd60a1a828 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:09.619857 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba7497-8b16-446d-85c8-18dd60a1a828 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:09.624441 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2662f-a931-4fd0-9e35-3955c43a37df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:09.625029 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2662f-a931-4fd0-9e35-3955c43a37df admin admin] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:09.625347 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2662f-a931-4fd0-9e35-3955c43a37df admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:09.625611 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2662f-a931-4fd0-9e35-3955c43a37df admin admin] PATH_INFO: `/v3/roles` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:09.626540 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2662f-a931-4fd0-9e35-3955c43a37df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:09.628124 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2662f-a931-4fd0-9e35-3955c43a37df admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:09.638553 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1454/2922] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:09.643115 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf38c5-a022-4ca8-97ea-e85890a419f3 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:09.653042 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf38c5-a022-4ca8-97ea-e85890a419f3 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:09.658631 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcd8a8-8841-4faa-8dc3-242e28314478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:09.659514 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcd8a8-8841-4faa-8dc3-242e28314478 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:09.659822 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcd8a8-8841-4faa-8dc3-242e28314478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:09.660097 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcd8a8-8841-4faa-8dc3-242e28314478 admin admin] PATH_INFO: `/v3/projects/2b477593590943b88325822f014f0411/users/e7f34fed804446a98762e11bce9d7b3c/roles/e385f7d11fac4932a910ec0ef95e2a43` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:09.671674 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:09.676952 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcd8a8-8841-4faa-8dc3-242e28314478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b477593590943b88325822f014f0411, user_id=e7f34fed804446a98762e11bce9d7b3c, role_id=e385f7d11fac4932a910ec0ef95e2a43)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:09.680156 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcd8a8-8841-4faa-8dc3-242e28314478 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:09.693493 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1469/2923] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:09 2022] PUT /identity/v3/projects/2b477593590943b88325822f014f0411/users/e7f34fed804446a98762e11bce9d7b3c/roles/e385f7d11fac4932a910ec0ef95e2a43 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:24:09.698797 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35d83d-8a81-4188-a99e-a280c1077fc9 None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:09.699082 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35d83d-8a81-4188-a99e-a280c1077fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:09.699357 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35d83d-8a81-4188-a99e-a280c1077fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:09.718341 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-eb35d83d-8a81-4188-a99e-a280c1077fc9 None None] MFA Rules not processed for user `5c92b48d12534a97833b0661f5d186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:09.720080 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-eb35d83d-8a81-4188-a99e-a280c1077fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:09.771684 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:09.783487 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1455/2924] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:09.793537 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcd8a8-8841-4faa-8dc3-242e28314478 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:09.795274 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-67fcd8a8-8841-4faa-8dc3-242e28314478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:09.854071 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcd8a8-8841-4faa-8dc3-242e28314478 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:09.857698 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c140a-a402-4635-b4a6-8194fec844b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:09.858326 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c140a-a402-4635-b4a6-8194fec844b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:09.858602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c140a-a402-4635-b4a6-8194fec844b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:09.858870 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c140a-a402-4635-b4a6-8194fec844b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:09.867749 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c140a-a402-4635-b4a6-8194fec844b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:09.869155 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c140a-a402-4635-b4a6-8194fec844b3 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:09.878723 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1470/2925] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:24:13.898240 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc69bc-6bb5-4fec-8bf5-90c352cd3e2d None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:13.898788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc69bc-6bb5-4fec-8bf5-90c352cd3e2d None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:13.898788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc69bc-6bb5-4fec-8bf5-90c352cd3e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:13.928396 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-29bc69bc-6bb5-4fec-8bf5-90c352cd3e2d None None] MFA Rules not processed for user `e7f34fed804446a98762e11bce9d7b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:13.929619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-29bc69bc-6bb5-4fec-8bf5-90c352cd3e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:13.997632 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1456/2926] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:24:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:14.003147 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c140a-a402-4635-b4a6-8194fec844b3 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:14.013879 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c140a-a402-4635-b4a6-8194fec844b3 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:14.019467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334470c8-2f49-43d2-9761-28302b05e199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:14.020085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334470c8-2f49-43d2-9761-28302b05e199 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:14.020365 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334470c8-2f49-43d2-9761-28302b05e199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:14.020631 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334470c8-2f49-43d2-9761-28302b05e199 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:14.021280 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334470c8-2f49-43d2-9761-28302b05e199 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:14.023258 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334470c8-2f49-43d2-9761-28302b05e199 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:14.053694 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1471/2927] 10.222.0.62 () {60 vars in 1153 bytes} [Fri Sep 9 12:24:14 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:14.057849 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc69bc-6bb5-4fec-8bf5-90c352cd3e2d None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:14.069942 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc69bc-6bb5-4fec-8bf5-90c352cd3e2d None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:14.129361 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761444a2-2546-4086-a262-25d123505acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:14.130424 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761444a2-2546-4086-a262-25d123505acc admin admin] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:14.130705 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761444a2-2546-4086-a262-25d123505acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:14.130980 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761444a2-2546-4086-a262-25d123505acc admin admin] PATH_INFO: `/v3/users` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:14.131657 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761444a2-2546-4086-a262-25d123505acc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:14.133612 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761444a2-2546-4086-a262-25d123505acc admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:14.157170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1457/2928] 10.222.0.62 () {60 vars in 1144 bytes} [Fri Sep 9 12:24:14 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:14.161379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334470c8-2f49-43d2-9761-28302b05e199 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:14.172479 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334470c8-2f49-43d2-9761-28302b05e199 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:14.177133 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a648c-ed45-447f-9530-7ee2b785f01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:14.177734 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a648c-ed45-447f-9530-7ee2b785f01c admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:14.178012 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a648c-ed45-447f-9530-7ee2b785f01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:14.178314 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a648c-ed45-447f-9530-7ee2b785f01c admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:14.178975 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a648c-ed45-447f-9530-7ee2b785f01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:14.180556 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a648c-ed45-447f-9530-7ee2b785f01c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:14.191676 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1472/2929] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:14.196161 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761444a2-2546-4086-a262-25d123505acc admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:14.207143 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761444a2-2546-4086-a262-25d123505acc admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:14.211897 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f0088-9c9e-4218-a029-a2e7f128bf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:14.212548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f0088-9c9e-4218-a029-a2e7f128bf46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:14.212826 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f0088-9c9e-4218-a029-a2e7f128bf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:14.213097 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f0088-9c9e-4218-a029-a2e7f128bf46 admin admin] PATH_INFO: `/v3/projects/38724251fdd244428ce92dfb19e4b366/users/da307d2025e84eba9c71034c6fb285a7/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:14.228993 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f0088-9c9e-4218-a029-a2e7f128bf46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38724251fdd244428ce92dfb19e4b366, user_id=da307d2025e84eba9c71034c6fb285a7, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:14.231631 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f0088-9c9e-4218-a029-a2e7f128bf46 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:14.244548 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1458/2930] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:14 2022] PUT /identity/v3/projects/38724251fdd244428ce92dfb19e4b366/users/da307d2025e84eba9c71034c6fb285a7/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:24:14.248576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a648c-ed45-447f-9530-7ee2b785f01c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:14.259567 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a648c-ed45-447f-9530-7ee2b785f01c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:14.316604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34697159-6f22-4dcd-8cfc-30bfb4d291b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:14.317212 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34697159-6f22-4dcd-8cfc-30bfb4d291b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:14.317491 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34697159-6f22-4dcd-8cfc-30bfb4d291b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:14.317764 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34697159-6f22-4dcd-8cfc-30bfb4d291b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:14.318447 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34697159-6f22-4dcd-8cfc-30bfb4d291b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:14.320035 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34697159-6f22-4dcd-8cfc-30bfb4d291b1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:14.329916 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1473/2931] 10.222.0.62 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:14.334100 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f0088-9c9e-4218-a029-a2e7f128bf46 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:14.345084 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f0088-9c9e-4218-a029-a2e7f128bf46 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:14.350178 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc550241-4bf1-43c3-9c99-157ba1fb8bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:14.350837 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc550241-4bf1-43c3-9c99-157ba1fb8bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:14.351113 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc550241-4bf1-43c3-9c99-157ba1fb8bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:14.351386 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc550241-4bf1-43c3-9c99-157ba1fb8bf9 admin admin] PATH_INFO: `/v3/projects/38724251fdd244428ce92dfb19e4b366/users/da307d2025e84eba9c71034c6fb285a7/roles/a57eb61267c6490c8cb77fb1e31926dc` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:14.353488 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc550241-4bf1-43c3-9c99-157ba1fb8bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38724251fdd244428ce92dfb19e4b366, user_id=da307d2025e84eba9c71034c6fb285a7, role_id=a57eb61267c6490c8cb77fb1e31926dc)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:14.356169 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc550241-4bf1-43c3-9c99-157ba1fb8bf9 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:14.367506 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1459/2932] 10.222.0.62 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:14 2022] PUT /identity/v3/projects/38724251fdd244428ce92dfb19e4b366/users/da307d2025e84eba9c71034c6fb285a7/roles/a57eb61267c6490c8cb77fb1e31926dc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:18.886802 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff674d3f-329f-42c6-ae8b-b257a650167d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:18.887380 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff674d3f-329f-42c6-ae8b-b257a650167d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:18.887917 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff674d3f-329f-42c6-ae8b-b257a650167d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:18.925880 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ff674d3f-329f-42c6-ae8b-b257a650167d None None] MFA Rules not processed for user `da307d2025e84eba9c71034c6fb285a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:18.927186 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ff674d3f-329f-42c6-ae8b-b257a650167d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:18.998750 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:19.031124 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1474/2933] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:24:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:19.047891 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc550241-4bf1-43c3-9c99-157ba1fb8bf9 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:19.050067 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-fc550241-4bf1-43c3-9c99-157ba1fb8bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:19.109719 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:19.159171 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc550241-4bf1-43c3-9c99-157ba1fb8bf9 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:19.165410 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00111d3d-171b-4004-890a-857705da9d7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:19.166040 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00111d3d-171b-4004-890a-857705da9d7a service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:19.166407 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00111d3d-171b-4004-890a-857705da9d7a service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:19.166679 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00111d3d-171b-4004-890a-857705da9d7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:19.251720 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00111d3d-171b-4004-890a-857705da9d7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:19.253351 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00111d3d-171b-4004-890a-857705da9d7a service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:19.265028 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1460/2934] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:29.027622 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=9 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:29.260290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=38 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:39.829846 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3705146-f014-4e0f-929f-d7790e13f3fa None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:39.830530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3705146-f014-4e0f-929f-d7790e13f3fa None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:39.830530 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3705146-f014-4e0f-929f-d7790e13f3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:39.859756 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b3705146-f014-4e0f-929f-d7790e13f3fa None None] MFA Rules not processed for user `e7f34fed804446a98762e11bce9d7b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:39.861146 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b3705146-f014-4e0f-929f-d7790e13f3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:39.886523 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1475/2935] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:24:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:40.212472 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00111d3d-171b-4004-890a-857705da9d7a service nova] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:40.288099 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00111d3d-171b-4004-890a-857705da9d7a service nova] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:40.291735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa59461-e65a-4a60-b52e-c8ae93b53005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:40.292334 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa59461-e65a-4a60-b52e-c8ae93b53005 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:40.292671 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa59461-e65a-4a60-b52e-c8ae93b53005 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:40.292945 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa59461-e65a-4a60-b52e-c8ae93b53005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:40.294704 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-cfa59461-e65a-4a60-b52e-c8ae93b53005 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:40.305037 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa59461-e65a-4a60-b52e-c8ae93b53005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:40.306496 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa59461-e65a-4a60-b52e-c8ae93b53005 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:40.314474 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1461/2936] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:41.762341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3705146-f014-4e0f-929f-d7790e13f3fa None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:41.763762 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b3705146-f014-4e0f-929f-d7790e13f3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:41.863832 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3705146-f014-4e0f-929f-d7790e13f3fa None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:41.869096 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36e7ec-e9a8-4b62-9be9-c277e2a6de51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:41.869898 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36e7ec-e9a8-4b62-9be9-c277e2a6de51 service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:41.870211 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36e7ec-e9a8-4b62-9be9-c277e2a6de51 service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:41.870489 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36e7ec-e9a8-4b62-9be9-c277e2a6de51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:41.872460 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3b36e7ec-e9a8-4b62-9be9-c277e2a6de51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:41.886994 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36e7ec-e9a8-4b62-9be9-c277e2a6de51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:41.888635 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36e7ec-e9a8-4b62-9be9-c277e2a6de51 service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:41.902698 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1476/2937] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:44.425252 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa59461-e65a-4a60-b52e-c8ae93b53005 service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:44.426671 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-cfa59461-e65a-4a60-b52e-c8ae93b53005 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:44.441616 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa59461-e65a-4a60-b52e-c8ae93b53005 service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:44.446544 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62c399-7dcc-4ebd-be39-f21dd7336597 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:44.447129 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62c399-7dcc-4ebd-be39-f21dd7336597 service placement] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:44.447402 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62c399-7dcc-4ebd-be39-f21dd7336597 service placement] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:44.447689 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62c399-7dcc-4ebd-be39-f21dd7336597 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:44.449300 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-bc62c399-7dcc-4ebd-be39-f21dd7336597 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:44.460601 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62c399-7dcc-4ebd-be39-f21dd7336597 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:44.462191 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62c399-7dcc-4ebd-be39-f21dd7336597 service placement] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:44.471958 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1462/2938] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:46.827216 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36e7ec-e9a8-4b62-9be9-c277e2a6de51 service placement] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:46.840730 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36e7ec-e9a8-4b62-9be9-c277e2a6de51 service placement] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:46.846715 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3dbc92-e079-4542-a9af-cbdf378cccb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:46.847322 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dbc92-e079-4542-a9af-cbdf378cccb6 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:46.847604 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dbc92-e079-4542-a9af-cbdf378cccb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:46.847875 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dbc92-e079-4542-a9af-cbdf378cccb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:46.849629 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ff3dbc92-e079-4542-a9af-cbdf378cccb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:46.863622 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3dbc92-e079-4542-a9af-cbdf378cccb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:46.865057 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3dbc92-e079-4542-a9af-cbdf378cccb6 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:46.878016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1477/2939] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:24:54.467749 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=41 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:56.871687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=65 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:58.158356 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62c399-7dcc-4ebd-be39-f21dd7336597 service placement] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:58.160068 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-bc62c399-7dcc-4ebd-be39-f21dd7336597 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:58.178380 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62c399-7dcc-4ebd-be39-f21dd7336597 service placement] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:58.184652 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff8325-741a-4b4f-8546-6779e2af2fc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:58.185275 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff8325-741a-4b4f-8546-6779e2af2fc2 service nova] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:58.185563 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff8325-741a-4b4f-8546-6779e2af2fc2 service nova] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:58.185835 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff8325-741a-4b4f-8546-6779e2af2fc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:58.187848 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-72ff8325-741a-4b4f-8546-6779e2af2fc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:58.203959 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff8325-741a-4b4f-8546-6779e2af2fc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:58.205732 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff8325-741a-4b4f-8546-6779e2af2fc2 service nova] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:58.217474 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1463/2940] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:25:00.743610 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3dbc92-e079-4542-a9af-cbdf378cccb6 service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:25:00.745016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ff3dbc92-e079-4542-a9af-cbdf378cccb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:25:00.781270 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3dbc92-e079-4542-a9af-cbdf378cccb6 service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:25:00.785274 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544049-a8f5-4125-8d7b-f781f8d8efb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:25:00.785907 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544049-a8f5-4125-8d7b-f781f8d8efb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:25:00.786231 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544049-a8f5-4125-8d7b-f781f8d8efb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:25:00.786531 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544049-a8f5-4125-8d7b-f781f8d8efb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:25:00.795614 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d544049-a8f5-4125-8d7b-f781f8d8efb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:25:00.797018 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d544049-a8f5-4125-8d7b-f781f8d8efb6 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:25:00.806529 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1478/2941] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:25:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:25:08.212478 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=6 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:25:10.802372 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=15 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:27:16.525027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff8325-741a-4b4f-8546-6779e2af2fc2 service nova] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:27:16.534569 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff8325-741a-4b4f-8546-6779e2af2fc2 service nova] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:27:16.538087 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20706d-def0-4873-a11e-f81a6c7603b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:27:16.538713 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20706d-def0-4873-a11e-f81a6c7603b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:27:16.538987 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20706d-def0-4873-a11e-f81a6c7603b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:27:16.539259 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20706d-def0-4873-a11e-f81a6c7603b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:27:16.541114 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-9b20706d-def0-4873-a11e-f81a6c7603b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:27:16.575448 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20706d-def0-4873-a11e-f81a6c7603b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:27:16.576870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20706d-def0-4873-a11e-f81a6c7603b6 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:27:16.597206 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1464/2942] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:27:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:27:17.791787 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544049-a8f5-4125-8d7b-f781f8d8efb6 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:27:17.800686 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544049-a8f5-4125-8d7b-f781f8d8efb6 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:27:17.804355 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9ef63-8808-4190-831a-d75bfc1c7d3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:27:17.804956 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9ef63-8808-4190-831a-d75bfc1c7d3f service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:27:17.805251 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9ef63-8808-4190-831a-d75bfc1c7d3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:27:17.805521 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9ef63-8808-4190-831a-d75bfc1c7d3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:27:17.807294 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bca9ef63-8808-4190-831a-d75bfc1c7d3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:27:17.821379 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9ef63-8808-4190-831a-d75bfc1c7d3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:27:17.822833 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9ef63-8808-4190-831a-d75bfc1c7d3f service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:27:17.834560 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1479/2943] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:27:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:27:21.249515 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065b2b8-851b-4ea7-8af5-785912b5429c None None] REQUEST_METHOD: `POST` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:27:21.250038 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065b2b8-851b-4ea7-8af5-785912b5429c None None] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:27:21.250170 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065b2b8-851b-4ea7-8af5-785912b5429c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:27:21.270310 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.auth.core [None req-0065b2b8-851b-4ea7-8af5-785912b5429c None None] MFA Rules not processed for user `e7f34fed804446a98762e11bce9d7b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81447) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:27:21.271510 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0065b2b8-851b-4ea7-8af5-785912b5429c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:27:21.278336 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1465/2944] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:27:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:27:23.924444 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9ef63-8808-4190-831a-d75bfc1c7d3f service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:27:23.933594 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9ef63-8808-4190-831a-d75bfc1c7d3f service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:27:23.937311 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2648812-3f11-45e7-89c8-c63ccdfc43d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:27:23.938029 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2648812-3f11-45e7-89c8-c63ccdfc43d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:27:23.938250 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2648812-3f11-45e7-89c8-c63ccdfc43d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:27:23.939246 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2648812-3f11-45e7-89c8-c63ccdfc43d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:27:23.940717 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b2648812-3f11-45e7-89c8-c63ccdfc43d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:27:23.950632 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2648812-3f11-45e7-89c8-c63ccdfc43d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:27:23.952037 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2648812-3f11-45e7-89c8-c63ccdfc43d4 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:27:23.960878 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1480/2945] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:27:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:27:31.264658 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=21 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:27:33.957016 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=12 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:29:28.186090 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065b2b8-851b-4ea7-8af5-785912b5429c None None] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:29:28.187574 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-0065b2b8-851b-4ea7-8af5-785912b5429c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:29:28.270065 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065b2b8-851b-4ea7-8af5-785912b5429c None None] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:29:28.274824 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd024b-efd4-44ee-9aad-4905646d292c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25411d51a524f4b9f0ff846a0b29b6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:29:28.275442 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd024b-efd4-44ee-9aad-4905646d292c service cinder] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:29:28.275722 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd024b-efd4-44ee-9aad-4905646d292c service cinder] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:29:28.275995 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd024b-efd4-44ee-9aad-4905646d292c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:29:28.286292 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd024b-efd4-44ee-9aad-4905646d292c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:29:28.287715 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd024b-efd4-44ee-9aad-4905646d292c service cinder] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:29:28.296796 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1466/2946] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:29:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:29:38.292736 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=35 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:29:39.333408 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60fc59-7007-45a7-822a-2eaf22a50cd9 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:29:39.333933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60fc59-7007-45a7-822a-2eaf22a50cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:29:39.333933 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60fc59-7007-45a7-822a-2eaf22a50cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:29:39.352703 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7f60fc59-7007-45a7-822a-2eaf22a50cd9 None None] MFA Rules not processed for user `e7f34fed804446a98762e11bce9d7b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:29:39.353903 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7f60fc59-7007-45a7-822a-2eaf22a50cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:29:39.360695 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1481/2947] 10.222.0.62 () {58 vars in 958 bytes} [Fri Sep 9 12:29:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:29:39.697591 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd024b-efd4-44ee-9aad-4905646d292c service cinder] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:29:39.706940 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd024b-efd4-44ee-9aad-4905646d292c service cinder] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:29:39.710423 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f47d6-e590-4907-8942-2236992f51a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:29:39.711009 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f47d6-e590-4907-8942-2236992f51a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:29:39.711278 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f47d6-e590-4907-8942-2236992f51a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:29:39.711545 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f47d6-e590-4907-8942-2236992f51a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:29:39.713222 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-174f47d6-e590-4907-8942-2236992f51a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:29:39.722635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f47d6-e590-4907-8942-2236992f51a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:29:39.724070 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f47d6-e590-4907-8942-2236992f51a7 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:29:39.731922 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1467/2948] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:29:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:29:41.192996 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60fc59-7007-45a7-822a-2eaf22a50cd9 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:29:41.205720 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60fc59-7007-45a7-822a-2eaf22a50cd9 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:29:41.210697 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47d987-d363-44f5-8a70-cefb3739a966 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:29:41.211298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47d987-d363-44f5-8a70-cefb3739a966 service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:29:41.211576 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47d987-d363-44f5-8a70-cefb3739a966 service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:29:41.211849 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47d987-d363-44f5-8a70-cefb3739a966 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:29:41.220670 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47d987-d363-44f5-8a70-cefb3739a966 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:29:41.222298 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47d987-d363-44f5-8a70-cefb3739a966 service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:29:41.232857 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1482/2949] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:29:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:29:49.728373 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=6 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:29:51.228385 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=9 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:30:01.405282 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f47d6-e590-4907-8942-2236992f51a7 service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:30:01.414775 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f47d6-e590-4907-8942-2236992f51a7 service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:30:01.418324 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1decc-04e5-4788-bc9a-0425af24d883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:30:01.418911 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1decc-04e5-4788-bc9a-0425af24d883 service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:30:01.419181 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1decc-04e5-4788-bc9a-0425af24d883 service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:30:01.419447 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1decc-04e5-4788-bc9a-0425af24d883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:30:01.428982 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1decc-04e5-4788-bc9a-0425af24d883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:30:01.430485 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1decc-04e5-4788-bc9a-0425af24d883 service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:30:01.439965 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1468/2950] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:30:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:30:03.288050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47d987-d363-44f5-8a70-cefb3739a966 service placement] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:30:03.304116 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47d987-d363-44f5-8a70-cefb3739a966 service placement] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:30:03.310548 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0ac79-7ddf-44c7-8735-34fa1722acd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:30:03.311165 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0ac79-7ddf-44c7-8735-34fa1722acd3 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:30:03.311444 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0ac79-7ddf-44c7-8735-34fa1722acd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:30:03.311716 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0ac79-7ddf-44c7-8735-34fa1722acd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:30:03.313467 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-83f0ac79-7ddf-44c7-8735-34fa1722acd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:30:03.326596 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0ac79-7ddf-44c7-8735-34fa1722acd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:30:03.328173 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0ac79-7ddf-44c7-8735-34fa1722acd3 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:30:03.339724 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1483/2951] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:30:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:30:11.435756 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=6 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:30:13.334997 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:30:34.608383 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1decc-04e5-4788-bc9a-0425af24d883 service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:30:34.621088 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1decc-04e5-4788-bc9a-0425af24d883 service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:30:34.626338 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf96c2-4fc3-4062-86ad-a42606821220 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:30:34.626990 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf96c2-4fc3-4062-86ad-a42606821220 service placement] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:30:34.627587 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf96c2-4fc3-4062-86ad-a42606821220 service placement] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:30:34.627794 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf96c2-4fc3-4062-86ad-a42606821220 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:30:34.637221 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf96c2-4fc3-4062-86ad-a42606821220 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:30:34.638907 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf96c2-4fc3-4062-86ad-a42606821220 service placement] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:30:34.650054 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1469/2952] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:30:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:30:44.645406 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=6 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:32:14.964906 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0ac79-7ddf-44c7-8735-34fa1722acd3 service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:32:14.979677 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0ac79-7ddf-44c7-8735-34fa1722acd3 service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:32:14.985438 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d61dfa-31ef-43ea-aff0-c797125b3f1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b30e80c421245b5a1ed5849b28ea90e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:32:14.986046 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d61dfa-31ef-43ea-aff0-c797125b3f1f service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:32:14.986377 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d61dfa-31ef-43ea-aff0-c797125b3f1f service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:32:14.986659 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d61dfa-31ef-43ea-aff0-c797125b3f1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:32:14.997957 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d61dfa-31ef-43ea-aff0-c797125b3f1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:32:14.999555 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d61dfa-31ef-43ea-aff0-c797125b3f1f service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:32:15.011238 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1484/2953] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:32:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:32:18.135094 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf96c2-4fc3-4062-86ad-a42606821220 service placement] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:32:18.144603 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf96c2-4fc3-4062-86ad-a42606821220 service placement] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:32:18.148742 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417b04d-717c-4b30-9829-cb2d7ab2981a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd00634cdd3fb41e794d3af0b88ba7338', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:32:18.149325 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417b04d-717c-4b30-9829-cb2d7ab2981a service neutron] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:32:18.149599 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417b04d-717c-4b30-9829-cb2d7ab2981a service neutron] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:32:18.149870 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417b04d-717c-4b30-9829-cb2d7ab2981a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:32:18.220764 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2417b04d-717c-4b30-9829-cb2d7ab2981a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:32:18.222164 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2417b04d-717c-4b30-9829-cb2d7ab2981a service neutron] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:32:18.241900 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1470/2954] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:32:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:32:21.290626 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d61dfa-31ef-43ea-aff0-c797125b3f1f service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:32:21.305878 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d61dfa-31ef-43ea-aff0-c797125b3f1f service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:32:21.313816 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057fdd7-f4dd-451f-92a5-b8f744641706 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:32:21.314608 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057fdd7-f4dd-451f-92a5-b8f744641706 service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:32:21.315134 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057fdd7-f4dd-451f-92a5-b8f744641706 service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:32:21.315666 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057fdd7-f4dd-451f-92a5-b8f744641706 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:32:21.318683 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9057fdd7-f4dd-451f-92a5-b8f744641706 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:32:21.335872 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057fdd7-f4dd-451f-92a5-b8f744641706 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:32:21.338219 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057fdd7-f4dd-451f-92a5-b8f744641706 service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:32:21.355341 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1485/2955] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:32:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:32:24.600863 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417b04d-717c-4b30-9829-cb2d7ab2981a service neutron] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:32:24.613203 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417b04d-717c-4b30-9829-cb2d7ab2981a service neutron] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:32:24.618554 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28903316-9261-4fa2-90b1-5937540371f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:32:24.619190 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28903316-9261-4fa2-90b1-5937540371f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:32:24.619466 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28903316-9261-4fa2-90b1-5937540371f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:32:24.619735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28903316-9261-4fa2-90b1-5937540371f2 admin admin] PATH_INFO: `/v3/users/e7f34fed804446a98762e11bce9d7b3c` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:32:24.620856 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28903316-9261-4fa2-90b1-5937540371f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f34fed804446a98762e11bce9d7b3c)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:32:24.622899 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28903316-9261-4fa2-90b1-5937540371f2 admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:32:24.670002 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-28903316-9261-4fa2-90b1-5937540371f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f34fed804446a98762e11bce9d7b3c'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:24.677871 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-28903316-9261-4fa2-90b1-5937540371f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f34fed804446a98762e11bce9d7b3c'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:24.689656 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-28903316-9261-4fa2-90b1-5937540371f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f34fed804446a98762e11bce9d7b3c'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:24.697139 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-28903316-9261-4fa2-90b1-5937540371f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f34fed804446a98762e11bce9d7b3c'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:24.697788 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-28903316-9261-4fa2-90b1-5937540371f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f34fed804446a98762e11bce9d7b3c'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:24.703575 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1471/2956] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:32:24 2022] DELETE /identity/v3/users/e7f34fed804446a98762e11bce9d7b3c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:32:24.874630 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057fdd7-f4dd-451f-92a5-b8f744641706 service placement] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:32:24.886468 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057fdd7-f4dd-451f-92a5-b8f744641706 service placement] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:32:24.943453 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1827f4-d7ce-485b-911f-aa632b807642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:32:24.944085 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1827f4-d7ce-485b-911f-aa632b807642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:32:24.944366 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1827f4-d7ce-485b-911f-aa632b807642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:32:24.944640 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1827f4-d7ce-485b-911f-aa632b807642 admin admin] PATH_INFO: `/v3/projects/2b477593590943b88325822f014f0411` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:32:24.945753 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1827f4-d7ce-485b-911f-aa632b807642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b477593590943b88325822f014f0411)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:32:24.947688 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1827f4-d7ce-485b-911f-aa632b807642 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:32:24.995045 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6c1827f4-d7ce-485b-911f-aa632b807642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b477593590943b88325822f014f0411', 'request_id': 'req-6c1827f4-d7ce-485b-911f-aa632b807642'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:25.002627 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6c1827f4-d7ce-485b-911f-aa632b807642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b477593590943b88325822f014f0411', 'request_id': 'req-6c1827f4-d7ce-485b-911f-aa632b807642'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:25.008088 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6c1827f4-d7ce-485b-911f-aa632b807642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b477593590943b88325822f014f0411', 'request_id': 'req-6c1827f4-d7ce-485b-911f-aa632b807642'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:25.030770 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6c1827f4-d7ce-485b-911f-aa632b807642 admin admin] The token cache is being invalidate because project 2b477593590943b88325822f014f0411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:32:25.031060 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6c1827f4-d7ce-485b-911f-aa632b807642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:25.031687 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6c1827f4-d7ce-485b-911f-aa632b807642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:25.033330 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1486/2957] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:32:24 2022] DELETE /identity/v3/projects/2b477593590943b88325822f014f0411 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:32:25.037821 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28903316-9261-4fa2-90b1-5937540371f2 admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:32:25.039270 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-28903316-9261-4fa2-90b1-5937540371f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:32:25.052635 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28903316-9261-4fa2-90b1-5937540371f2 admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:32:25.095242 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:32:25.108387 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc0204-5c74-478a-8412-3efcfcc3547d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:32:25.109016 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc0204-5c74-478a-8412-3efcfcc3547d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:32:25.109290 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc0204-5c74-478a-8412-3efcfcc3547d admin admin] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:32:25.109557 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc0204-5c74-478a-8412-3efcfcc3547d admin admin] PATH_INFO: `/v3/users/da307d2025e84eba9c71034c6fb285a7` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:32:25.110711 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc0204-5c74-478a-8412-3efcfcc3547d admin admin] RBAC: Authorizing `identity:delete_user(user_id=da307d2025e84eba9c71034c6fb285a7)` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:32:25.112608 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc0204-5c74-478a-8412-3efcfcc3547d admin admin] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:32:25.160737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fdbc0204-5c74-478a-8412-3efcfcc3547d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da307d2025e84eba9c71034c6fb285a7'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:25.169247 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fdbc0204-5c74-478a-8412-3efcfcc3547d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da307d2025e84eba9c71034c6fb285a7'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:25.181131 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fdbc0204-5c74-478a-8412-3efcfcc3547d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da307d2025e84eba9c71034c6fb285a7'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:25.187948 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fdbc0204-5c74-478a-8412-3efcfcc3547d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da307d2025e84eba9c71034c6fb285a7'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:25.188608 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.notifications [None req-fdbc0204-5c74-478a-8412-3efcfcc3547d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da307d2025e84eba9c71034c6fb285a7'} {{(pid=81447) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:25.194581 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1472/2958] 10.222.0.62 () {58 vars in 1224 bytes} [Fri Sep 9 12:32:25 2022] DELETE /identity/v3/users/da307d2025e84eba9c71034c6fb285a7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:32:25.372943 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1827f4-d7ce-485b-911f-aa632b807642 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:32:25.374554 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6c1827f4-d7ce-485b-911f-aa632b807642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:32:25.389731 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1827f4-d7ce-485b-911f-aa632b807642 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:32:25.458428 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:32:25.474369 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42af026-355d-4e70-a915-fe46526d86d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c92b48d12534a97833b0661f5d186ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7d1fa6c42548518e72832336a573e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:32:25.475050 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42af026-355d-4e70-a915-fe46526d86d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:32:25.475330 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42af026-355d-4e70-a915-fe46526d86d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:32:25.475602 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42af026-355d-4e70-a915-fe46526d86d3 admin admin] PATH_INFO: `/v3/projects/38724251fdd244428ce92dfb19e4b366` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:32:25.476752 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42af026-355d-4e70-a915-fe46526d86d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38724251fdd244428ce92dfb19e4b366)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:32:25.478692 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42af026-355d-4e70-a915-fe46526d86d3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:32:25.525477 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c42af026-355d-4e70-a915-fe46526d86d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38724251fdd244428ce92dfb19e4b366', 'request_id': 'req-c42af026-355d-4e70-a915-fe46526d86d3'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:25.532661 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c42af026-355d-4e70-a915-fe46526d86d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38724251fdd244428ce92dfb19e4b366', 'request_id': 'req-c42af026-355d-4e70-a915-fe46526d86d3'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:25.538187 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c42af026-355d-4e70-a915-fe46526d86d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38724251fdd244428ce92dfb19e4b366', 'request_id': 'req-c42af026-355d-4e70-a915-fe46526d86d3'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:25.548994 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c42af026-355d-4e70-a915-fe46526d86d3 admin admin] The token cache is being invalidate because project 38724251fdd244428ce92dfb19e4b366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:32:25.549214 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c42af026-355d-4e70-a915-fe46526d86d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:25.549834 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c42af026-355d-4e70-a915-fe46526d86d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:32:25.551315 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1487/2959] 10.222.0.62 () {58 vars in 1233 bytes} [Fri Sep 9 12:32:25 2022] DELETE /identity/v3/projects/38724251fdd244428ce92dfb19e4b366 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:32:35.191027 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:32:35.546072 dv-cinder-852210-13-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:35:39.583046 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc0204-5c74-478a-8412-3efcfcc3547d admin admin] Authenticating user token {{(pid=81447) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:35:39.584429 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-fdbc0204-5c74-478a-8412-3efcfcc3547d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:35:39.720821 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc0204-5c74-478a-8412-3efcfcc3547d admin admin] Validating token access rules against request {{(pid=81447) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:35:39.778735 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad09a74-724d-4be7-9026-f4ecaab2a76c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c58bae3ef7784671ae247f4ad4199c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32b06df0c5504592b012585f193386e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81447) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:35:39.779343 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad09a74-724d-4be7-9026-f4ecaab2a76c service placement] REQUEST_METHOD: `GET` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:35:39.779619 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad09a74-724d-4be7-9026-f4ecaab2a76c service placement] SCRIPT_NAME: `/identity` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:35:39.779888 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad09a74-724d-4be7-9026-f4ecaab2a76c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81447) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:35:39.781637 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.fernet_utils [None req-fad09a74-724d-4be7-9026-f4ecaab2a76c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81447) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:35:39.792737 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad09a74-724d-4be7-9026-f4ecaab2a76c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:35:39.794329 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad09a74-724d-4be7-9026-f4ecaab2a76c service placement] RBAC: Authorization granted {{(pid=81447) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:35:39.804058 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: [pid: 81447|app: 0|req: 1473/2960] 10.222.0.62 () {60 vars in 1365 bytes} [Fri Sep 9 12:35:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:35:49.799862 dv-cinder-852210-13-iscsi devstack@keystone.service[81447]: DEBUG dbcounter [-] [81447] Writing DB stats keystone:SELECT=89 {{(pid=81447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}